10 Useful Linux Commands For Programmers and Advanced Users
Linux is getting more and more users daily. Throughout the years it has steadily become one of the best choices in OS, whether it’s about casual users or hardcore programmers. On the surface, Linux, especially with the Ubuntu distribution, has made a great effort in standardizing Linux as a close rival of Windows and Mac OS. However, Linux is not just about a nice GUI, stability and security. It allows for more advanced usage, that can deeply help you with everyday tasks and projects, especially if you are a computer programmer. In this tutorial, we are going to be discussing a list of 10 very important linux tips that can greatly help the life of programmers and advanced Linux users.
10. Tail -f : View contents of log files while they are generated
Have you ever been in a scenario where a program is constantly creating new log file entries every second or so ? If not, you may want to take a look at /var/log/syslog/, which is your system’s log file. It’s very common that we need to look at the most current file entries as they are generated. It could be that we have created a program that logs events to a file and we want to be looking at those entries, for debugging purposes. Say that this logfile is named ‘log.txt’ and is generated on our home directory. We can execute the tail command to get the last lines of that file. However, if we do that, we will get a static representation of these lines, exactly at the time we executed the command.Luckily, if we execute it like :
tail -f ~/log.txt
we will cause the tail command to not stop when the end of file(EOF) is reached, but rather wait for additional input data. Remember this command, it will definitely come in handy.
9. The Translate Command – Caesar cipher in a one liner, capitalize and other deeds
The Unix command ‘tr’ (translate), is a really useful way to translate characters from one alphabet to another.It’s very simple and very powerful. This is how you can capitalize a string :
echo hello there | tr "a-z" "A-Z" HELLO THERE
Notice that this changes a downcase character of the alphabet a-z and maps it to the alphabet A-Z, thus capitalizing the string. Let’s look at a more advanced example, where we want to encrypt a string with a simple ROT-3, where a = d, b = e and so on :
echo hello there | tr "a-z" "d-za-c" khoor wkhuh
Notice that we first start with mapping the letters to d-z and when we reach z, we specify a new set of a-c for the remaining letters.
8. The nohup Command – Never let the ssh session bring down your process
The nohup command is not a well known one, but it is really useful in cases. The idea is that you can spawn a process that does not hangup even if you execute it remotely and you disconnect from an ssh connection. What it does is actually force a process to ignore the SIGHUP signal. The standard output of the command is saved in nohup.out in the current directory and it’s pretty convenient for later analysis.
You may be thinking where this can be useful. In my case, it has become quite useful when having to execute processes that take a few hours to finish(due to long processing). I want to execute these processes in a remote computer where I connect to via SSH. However, when i disconnect from the server, the processes that have been spawned are normally killed. I would like to keep them alive even after i disconnect and the way to do it is the nohup command. Let’s say that I want to execute a python script, this is how i would do it with nohup :
nohup python the_python_script.py &
This will force the process to execute even after disconnecting from the server and it will create a nohup.out file that contains the process standard output.
7. The mysterious 2>&1. A tale from the crypt.. or a better way to log ?
When you execute a process, there are different types of logging that can happen. The program is most probably using the standard output(stdout) and the standard error(stderr) for logging stuff. There are times where we want to redirect the messages to a file but we want to redirect both streams and not just one. The number 1 refers to stdout and the number 2 refers to stderr. As you most probably already know, the > token is a redirection command. But what really happens in the expression “2>&1” ? And how can you remember it ?
2>1 means something like redirect the standard error to a file that is named ‘1’. However, we actually want to use a file descriptor and not a file, since stdout is not really a file. Thus the & is used. Therefore :
cat file.txt 1 > log.txt cat file.txt 2 > log2.txt cat file.txt 2>&1 > log3.txt
The first command redirects the standard output to log.txt. The second redirects the standard error to log2.txt. The third one redirects the standard error to the standard output and then redirects the whole thing to log3.txt, thus capturing both streams.
6. Scp – Secure copying files remotely
Scp(secure copy) is a command that you cannot afford to miss if you are connecting to remote servers. Scp is a secure way to copy files to remote computers. The file exchange happens via ssh, which makes it pretty secure. It’s a good idea to have setup your ssh keys before doing that. Scp can be intimidating for new users, but it’s actually pretty easy. You have to remember the simple rule that the source is specified first. Personally i remember it because I am using it pretty much daily, but when I first learned about the command, I was using a trick to remember the order. I was thinking “scp source target” and because s comes before t in the alphabet, I knew that source has to come first
Now, sometimes the source is on the local computer and sometimes it is not. Let’s see both occasions :
scp my_local_file user@remote_ip:./files scp user@remote_ip:./files/specific_file /home/user/
The first command copies a local file to a remote pc. Notice that we use a : after the remote pc ip and then specify the path to save to, relative to the user(an absolute path can also be specified). The second command does the opposite thing. It copies a remote file to our local machine. Notice that both commands are meant to be execute from our local computer to exchange files to and from a remote pc. Remember that to copy directories, you need to use the -r switch right after scp like “scp -r local_directory user@remote_ip:./files”. Also, user@ can be avoided if your local bash user is the same as the remote user. Scp automatically syncs the user specification.
5. Netstat – The power of knowing what and who connects to your computer
Netstat is a very versatile command that cannot be covered in a full post, not even touched in a tip section. However, the idea is to let you know of the basic functionality so that you can dig deeper and learn more about it in your own pace. The netstat command is a network tool that can give you lots and lots of information about your system. A very standard usage is looking at the tcp/udp ports and their status in your machine. The very standard way to use it is like :
The switch -a shows all sockets and -n shows network addresses as numbers(you will get port 80 instead of http for instance). By executing this command you can see the connections that you have initiated or that others have initiated for your machine, as well as the connection ports, ips and state of the sockets(listening, established and so on). It’s a really handy tool.
4. Iconv – Convert encoding of files
There may be times where you need to convert a file from one character set to another. There have been times where i wanted to convert greek to utf8 for example. Especially if you are moving linux/mac os files to Windows. The conversion is actually quite simple to do with iconv and works like a charm. You just use the -t and -f switches to specify encoding FROM(-f) one character set TO(-t) another, and also specify the file input and output. Let’s take a look at the example below :
iconv -f iso8859-7 -t utf-8 input_file.txt > output_file.txt
The command above onverts the characters in file “input_file.txt” from iso8859-7 to utf-8 and saves the result to the file “output_file.txt”. Simple, right ?
3. Everything about Disk Usage using the powerful du command
Not many unix users know how to get disk usage information using their shell. There is a great tool that you can use to get this information. The great command du(disk usage) can give you a nice overview of how much your disk is used. It can be a bit difficult to remember the actual command with the switches, but it’s actually really easy once you understand what those stand for. By far the most standard usage is with the switch -sh. Let’s take a look :
The command seems like gibberish, but it actually means “Disk Usage in Summarized Human readable format”. That’s all it really means. If you cannot remember it, just thing of disk usage(du) and then the command “sh” or the word “shell”. This way, “du -sh” will come naturally to you. You can also get the size of a specific folder by specifying it after the switches, like :
du -sh /home/test_user
2. Escape + DOT. A really unknown shortcut that really really helps.
This is a very unknown shortcut on the bash shell(i’m not that sure if it exists on other shells, but the vast majority definitely uses bash). It’s a very powerful combination of keys that really pays to remember and use all the time. What escape + DOT does(literally pressing first ESC and then .) is repeat the last entered bash shell string. To make it clearer, think of a situation where you copy a file form the current path to the path “/home/test_user/my_folder/other_folder/more_folders/end_folder”. Now, this is what you do to copy the file, say ‘test.txt’ :
cp test.txt /home/test_user/my_folder/other_folder/more_folders/end_folder
And of course press enter to actually perform the copy. Now say that you want to cd to that folder, the end_folder. What do you do ? Do you copy paste the path ? Or maybe start rewriting it and press tab midways to autocomplete as much as possible ? The latter seems to be the better of the two methods, but there is a far superior method. Pressing ESC and then dot(.). This actually writes the whole path for you ! It just uses the last string entered and pastes it for you once you press the hotkey. Amazing, right ? It’s one of the hotkeys i use all the time. Two more to come in the next and final tip.
1. CTRL + A, CTRL + E and CTRL + U : Use the shell the way it’s supposed to be used
For the last tip, I included some more fancy and extremely useful shortcuts for the shell typers.
CTRL + A is used to go to the beginning of the shell command. If you have written a full command and your cursor is in the end, but you spotted a mistake in the beginning, press CTRL + A to move the cursor to the very beginning of your command.
CTRL + E does the exact opposite. Takes you to the end of the command. So if you spot a mistake, press CTRL+A, fix it and then CTRL+E to go back to where you were. Incredibly efficient for sure.
CTRL + U deletes the characters that precede the cursor. So if you want to delete a command, press CTRL + E to move to its end and then CTRL +U to delete it.
There are many more shortcuts like CTRL + W to just delete a word, that can make your life easier. I will stick to those for this post, since they are by far the ones that i use most and I believe that you will find them useful.
The End of the Post
Please leave a comment to let our readers know what your best tips are and please share this article with other people that enjoy linux/unix and would find these tips useful !