In this ultra-quick note I am going to study what are the Pipelines in Bash.
Before starting, and as a self-publicity, this text is part of my “Minimal guide to start programming in Bash”. If you are interested in reading a beginner in programming to learn about the subject, this is the right place.
But it is also important to keep in mind that this aspect of the terminal is not only useful when programming in Bash. Almost any command can be enriched by its use.
So without further ado:
What are Pipelines in Bash?
Well, after a quick search in the dictionary I now know that, among other things, the word “pipeline” in English means a pipe.
And since I’m talking about pipelines and the shell, I can guess that the terminal allows me to create pipelines.
To tell the truth, that doesn’t explain much… I’d better check the Bash manual. It says the following:
A pipeline is a sequence of one or more commands separated by one of the control operators ‘|’ or ‘|&’.
BASH MANUAL
Again, I can still intuit that a pipeline is a pipe that connects one or more commands.
But how does it connect them?
If I check it superficially, what I am going to find is something like “two commands piped” by a symbol. In the terminal it looks like this:
[time [-p]] [!] command1 [ | or |& command2 ] ...
Or to simplify:
command1 | command2
The symbol | (vertical bar) is one of the two symbols I can use to create a pipeline.
What do I get out of it?
The Bash manual elaborates with the following paragraph:
The output of each command in the pipeline is connected via a pipe to the input of the next command. That is, each command reads the previous command’s output.
BASH HANDBOOK
That is, using pipelines I get that the output (the result) of a first command is transformed into the input (the data to enter) in command that follows.
Thought in this way, they mean that the information travels from one side to the other. From one command to another.
I will prove it with an example.
On the one hand I have the “ls” command, which gives me a list of the files in a directory.
~$ ls Desktop Music Pictures Public Videos
On the other hand, the command “sort -R” randomly sorts any input I give it. What happens if I add it with a pipeline to the previous command?
This can be typed and reviewed directly in the terminal:
ls | sort -R Public Music Videos Pictures Desktop
By doing this, I get to redirect the result of “ls” (the names of the files in the directory) so that information becomes the input to the “sort” command. This is going to deliver me a random unordered list of the file names in the folder.
Granted, it may not be the most practical example… but you can’t deny that it works to see what’s going on.
It is worth adding that I can use multiple pipelines in a row, chaining the input and output of information between multiple commands.
For example, the “cat” command with the “-n” option (“-number”) adds a number to each line we give it as input. If I add it by a pipeline to the previous example, I am left with the following:
ls | sort -R | cat -n 1 Videos 2 Public 3 Desktop 4 Music 5 Pictures
In this way I can chain the input and output of several commands at the same time.
Some other variants for a pipeline:
I mentioned earlier that you can create a pipeline with a vertical slash “|”, but you can also do it with “|&”… What’s the difference?
The bash manual says:
If ‘|&’ is used, command1’s standard error, in addition to its standard output, is connected to command2’s standard input through the pipe (…)
If I make a pipeline with a vertical bar “|”, the chain is cut off at the command where the error occurs.
For example to try to get a simple error, if I type the command “touch” in the terminal I get this as a result:
touch touch: missing file operand Try 'touch --help' for more information.
That is, the command “touch” is missing information and I can look for more information about how it works with ‘touch –help’.
Now if I try to string this together:
touch | cat --number touch: missing file operand Try 'touch --help' for more information.
I can notice that the pipeline is terminated when the first error appears and the “cat -n” command is never executed.
But in the opposite case, if I perform a pipeline by the means of “|&”, the chain does not stop at the command where the error occurs. For example:
touch |& cat --number 1 touch: missing file operand 2 Try 'touch --help' for more information.
In thiscase, the pipeline continued through the error and went to the next command. By doing this, “cat -number” added a line number to each line of the entry.
Conclusion:
This is a topic that comes up many times when we study about Linux. So I hope you find it useful if you are looking for knowing what are the Pipelines in Bash.
I am always trying to improve the text, so I try to expand it as I learn new things. And if you find any errors in the text or examples, please let me know so I can correct them.
We’ll follow it up in the next entry.
