The Basics of Linux File Management


Knowing how to navigate within the Linux operating system and operate with files in the file system is the most important skill a future DevOps needs. Without that basic knowledge, any future endeavor within any Linux distro is going to be extremely hard. If one is not able to copy or move a file, create a directory or even view the contents of a directory, then he/she will lose their bearings quickly. So before you go deeper into the world of Linux, here are a few pointers on how to get about in it.

Creating, Moving, Copying, Removing and Finding Files

First off - before you can start making duplicates of files or move them around, you need to create some to begin with. Creating a file in Linux is a simple matter of “touching” it to existence. You can use the touch command to create an empty file by adding the file location and name after it. Creating an empty directory is as easy, just use the mkdir (which comes from “make directory”) command instead of touch. To view your freshly created file and directory you can use the ls command, which stands for “list”. ls has a bunch of useful parameters, the most common being -l which stands for a long list, meaning a more detailed view of the files and directories in alphabetical order. To move a file or directory to a different location just use the mv command. By the way this last command can also be used to rename files and directories. To create a copy of a file or directory the cp command is best. If you need to clean up some old files, then rm (which stands for “remove”) is the command for you.

Examples:

touch /var/myfirstfile		# this creates a file named myfirstfile into the /var directory
mkdir /myfirstdirectory		# this creates a directory named myfirstdirectory into the / directory 
ls -l /var			# shows the detailed contents of the /var directory 
ls -d /myfirstdirectory 	# shows the /myfirstdirectory (not its contents)
mv /var/myfirstfile /opt/myfirstfile	# this moves the file from the /var directory into the /opt directory
mv /var/myfirstfile /var/myfirstfilerenamed	# this renames the /var/myfirstfile to /var/myfirstfilerenamed
cp /var/myfirstfile /opt/	# this creates a copy of /var/myfirstfile into the /opt directory
rm /var/myfirstfile 		# this removes a file named myfirstfile

The powerful find command can be used to locate any files and directories from almost anywhere in the file system. Note that it doesn’t matter whether the search string is in the beginning, middle or end of the file name. This is made possible due to using the * and ' symbols in the command. Not only that, but it can also be used to find AND further send commands to the files it found by adding the -exec parameter to it. Another powerful command is the grep command which searches for patterns within a file itself.

Examples:

find / -name '*searchstring*' 			# this searches the file system for a file which includes "searchstring" in its name
find / -name '*searchstring*' -exec rm {} \; 	# this searches the file system for a file which includes "searchstring" in its name and deletes it with the rm command, the semicolon symbolizes the end of -exec section
grep searchstring /var/myfirstfile		# this searches for the pattern 'searchstring' from within /var/myfirstfile

File Permissions

Depending on the organization, you might need to keep certain people from accessing a file. This is where files permissions come into play. There are three basic types of permissions depending on whether we’re dealing with a file or a directory. Those permissions are read, write and execute.

File permissions are used as follows:

  • Read permissions allow to view the contents of the file;
  • Write permissions allow to create a new file, modify existing and delete the file;
  • Execute permissions allow to execute the file as a script.

Directory permissions are used as follows:

  • Read permissions allow to list the contents of the folder with ls command;
  • Write permissions allow to create a new file or subfolder, modify existing ones and delete files or folders;
  • Execute permissions allow to change the directory into this folder.

File and directory permissions can be viewed with the ls -l command where user permissions are divided into three sections in the following format -rwxrwxrwx, where…

  • the first position is reserved for file or directory indication;
  • second, third and fourth position (rwx) is reserved for the user;
  • fifth, sixth and seventh position (rwx) is reserved for the group;
  • and eight, ninth and tenth position (rwx) is reserved for anyone other than the user or group.

These permissions can be set with the help of chmod command and in two separate formats: numerical and alphabetical. In the numerical format, each permission has a determined number, so that if you add these numbers together you get the correct user, group and others’ permissions.

  • Read permission is equivalent to number 4;
  • Write permission is equivalent to number 2;
  • Execute permission is equivalent to number 1.

In order to apply new permissions, you’d have to calculate the read, write and execute permissions for all three interest groups. The same can be done with the alphabetical format, where permissions are assigned to using the same letters which the permissions and interest groups have. Adding permissions is done using chmod command. Using the numerical format DOES NOT take into account what permissions were present before the command, but the alphabetical format DOES. The numerical format is a fixed permission setting and the alphabetical is flexible.

Numerical format explanation


Octal Mode Number Description
0400 Allows the owner to read
0200 Allows the owner to write
0100 Allows the owner to execute files and search in the directory
0040 Allows group members to read
0020 Allows group members to write
0010 Allows group members to execute files and search in the directory
0004 Allows everyone in the world to read
0002 Allows everyone in the world to write
0001 Allows everyone in the world to execute files and search in the directory
1000 Sets the sticky bit
2000 Sets the setgid bit
4000 Sets the setuid bit

A setUID bit can be set only on a file and it makes the file executable in the user’s permissions. This means that if I have a file that can be executed as the root user, then I as an ordinary user will also be able to execute the same file in root permissions. This special permission is not widely used due to security reasons, but one specific place is the /usr/bin/passwd file that allows all the users to change their password without root permissions.

A setGID bit can be set on both files and directories, although it’s more commonly used with directories. When the setGID bit is added to a file all the system users can execute the file in the group permissions, but if the same permission is set to a directory all the files created in that directory automatically get the parent directory’s group permissions. Meaning that all the files created in that directory become automatically accessible to all the users who are within the parent directory’s group.

All the special permissions are set with the chmod command in both numerical and alphabetical format.

  • A Sticky bit is marked with the letter t in the others section and can be assigned as o+t or as number 1 in the first position before ordinary permissions (1xxx).
  • A SetUID bit is marked with the letter s in the user section and can be assigned as u+s or as number 4 in the first position before ordinary permissions (4xxx).
  • A SetGID bit is also marked with the letter s in the group section and can be assigned as g+s or as number 2 in the first position before ordinary permissions (2xxx).

Examples:

chmod o+t /myfirstdirectory 	# Sets a sticky bit to the directory (alphabetical)
chmod 1755 /myfirstdirectory 	# Sets a sticky bit to the directory (numerical)
chmod g+s /myfirstdirectory  	# Sets a setGID on the directory

The find command can be used once more with the help of the -perm parameter to find out whether there’s a setUID and sticky bit type of special permissions in the server.

Examples:

find / -perm /1000 	# find sticky bit files
find / -perm /2000 	# find setGID bit files
find / -perm /4000 	# find setUID bit files

File Ownership

Every file has an owner and a group assigned to it. By using the already familiar ls -l command you can find out the owner and group of a file. The owner is usually the one who created the file and can have almost absolute power over it, depending on the permissions. The group has separate permissions as it can consist of multiple users. All the users belonging to the same group have the same permissions to a file. Changing the owner or group of a file can be easily accomplished with the chown and chgrp commands.

Examples:

chown john /var/myfirstfile		# sets the user john as the file owner
chown john:johnsons /var/myfirstfile 	# sets the user john as the file owner and the group johnsons as the group
chgrp johnsons /var/myfirstfile 	# sets the group johnsons as the group

Compressing Files With TAR

It doesn’t matter if you’re managing a server or using a personal computer, disk space is a limited commodity. To minimize disk space usage or the time needed to move files from one location to another (especially over the internet), file compression comes into play. A compressed file is a sort of archive that contains one or more files that have been reduced in size. A common tool for compressing files is TAR (the name comes from “Tape Archive” because in the old days it was used to compress files to be stored on magnetic tape). TAR has a bunch of parameters that no-one seems to be able to remember by heart: -c is used to create an archive, -x is used to extract from an archive, -z is used to filter the archive through gzip (if this parameter is not used, then the files are not compressed but just moved to an archive), -v is used to verbose the output and list the files which are processed and -f to use archive file or device ARCHIVE. There are many more parameters for TAR of course (all visible in the man page of the command), but the ones listed here are the ones commonly used.

Examples:

tar -czvf myfirst.tar.gz /var/myfirstdirectory 	# compress the /var/myfirstdirectory directory into the archive named myfirst.tar.gz
tar -xzvf myfirst.tar.gz 			# extract the archive named myfirst.tar.gz

Conclusion

Managing files is only the first step into the world of Linux CLI and therefore easy to understand. These small commands become mighty tools in the hands of a skilled Linux administrator later on.

Roland Kaur