How to Read a File Line by Line in Bash Shell Script
It's pretty easy to read the contents of a Linux text file line by line in a crush script—as long as you bargain with some subtle gotchas. Hither's how to do it the safety way.
Files, Text, and Idioms
Each programming language has a fix of idioms. These are the standard, no-frills ways to attain a set of common tasks. They're the unproblematic or default fashion to utilize one of the features of the language the programmer is working with. They get role of a developer'southward toolkit of mental blueprints.
Deportment like reading data from files, working with loops, and swapping the values of two variables are practiced examples. The programmer will know at least one way to achieve their ends in a generic or vanilla fashion. Perchance that will suffice for the requirement at hand. Or maybe they'll embellish the code to arrive more efficient or applicable to the specific solution they are developing. But having the building-cake idiom at their fingertips is a great starting point.
Knowing and understanding idioms in 1 linguistic communication makes it easier to pick up a new programming language, besides. Knowing how things are constructed in ane language and looking for the equivalent—or the closest thing—in another language is a expert way to appreciate the similarities and differences between programming languages you already know and the one you lot're learning.
Reading Lines From a File: The One-Liner
In Fustigate, you can use a while loop on the command line to read each line of text from a file and do something with information technology. Our text file is called "data.txt." It holds a listing of the months of the year.
January February March . . October November Dec
Our simple i-liner is:
while read line; do echo $line; done < information.txt
The while loop reads a line from the file, and the execution period of the piffling programme passes to the body of the loop. The repeat command writes the line of text in the terminal window. The read attempt fails when there are no more lines to be read, and the loop is done.
1 neat trick is the power to redirect a file into a loop. In other programming languages, yous'd demand to open the file, read from it, and close information technology again when you lot'd finished. With Bash, y'all tin simply utilise file redirection and let the shell handle all of that low-level stuff for you.
Of class, this one-liner isn't terribly useful. Linux already provides the cat command, which does exactly that for us. Nosotros've created a long-winded way to replace a three-letter command. Only it does visibly demonstrate the principles of reading from a file.
That works well enough, up to a betoken. Suppose nosotros accept another text file that contains the names of the months. In this file, the escape sequence for a newline grapheme has been appended to each line. Nosotros'll call it "data2.txt."
January\n Feb\n March\due north . . Oct\northward November\northward December\n
Let'southward employ our ane-liner on our new file.
while read line; do echo $line; done < data2.txt
The backslash escape character " \ " has been discarded. The result is that an "n" has been appended to each line. Bash is interpreting the backslash equally the showtime of an escape sequence. Ofttimes, we don't want Bash to interpret what it is reading. It can be more convenient to read a line in its entirety—backslash escape sequences and all—and cull what to parse out or replace yourself, within your own code.
If we want to practise whatever meaningful processing or parsing on the lines of text, we'll need to use a script.
Reading Lines From a File With a Script
Here's our script. It'south called "script1.sh."
#!/bin/bash Counter=0 while IFS= '' read -r LinefromFile || [[ -n " ${LinefromFile} " ]]; practice (( Counter ++ )) repeat "Accessing line $Counter : ${LinefromFile} " done < " $1 " Nosotros set a variable called Counter to cipher, then we define our while loop.
The showtime argument on the while line is IFS='' . IFS stands for internal field separator. It holds values that Bash uses to identify give-and-take boundaries. By default, the read command strips off leading and trailing whitespace. If we want to read the lines from the file exactly as they are, we need to set IFS to be an empty string.
We could set this one time outside of the loop, simply like nosotros're setting the value of Counter . But with more complex scripts—especially those with many user-divers functions in them—it is possible that IFS could be ready to unlike values elsewhere in the script. Ensuring that IFS is ready to an empty string each time the while loop iterates guarantees that nosotros know what its beliefs volition be.
We're going to read a line of text into a variable chosen LinefromFile . We're using the -r (read backslash as a normal character) choice to ignore backslashes. They'll be treated just like whatever other character and won't receive any special treatment.
In that location are two conditions that will satisfy the while loop and allow the text to be processed by the torso of the loop:
-
read -r LinefromFile: When a line of text is successfully read from the file, thereadcontrol sends a success signal to thewhile, and thewhileloop passes the execution flow to the torso of the loop. Note that thereadcommand needs to see a newline character at the end of the line of text in order to consider it a successful read. If the file is not a POSIX compliant text file, the terminal line may not include a newline grapheme. If thereadcommand sees the stop of file marker (EOF) earlier the line is terminated by a newline, it will not treat information technology equally a successful read. If that happens, the last line of text will not be passed to the body of the loop and volition not be processed. -
[ -northward "${LinefromFile}" ]: Nosotros need to do some extra work to handle non-POSIX compatible files. This comparing checks the text that is read from the file. If it isn't terminated with a newline character, this comparing will still render success to thewhileloop. This ensures that whatsoever abaft line fragments are processed past the trunk of the loop.
These two clauses are separated by the OR logical operator " || " then that ifeither clause returns success, the retrieved text is processed by the body of the loop, whether there is a newline character or not.
In the body of our loop, nosotros're incrementing the Counter variable by ane and using echo to send some output to the final window. The line number and the text of each line are displayed.
We tin notwithstanding apply our redirection flim-flam to redirect a file into a loop. In this case, we're redirecting $i, a variable that holds the name of the first control line parameter that passed to the script. Using this pull a fast one on, we tin hands pass in the proper noun of the data file that nosotros want the script to piece of work on.
Re-create and paste the script into an editor and salve it with the filename "script1.sh." Use the chmod command to make information technology executable.
chmod +10 script1.sh
Let's run into what our script makes of the data2.txt text file and the backslashes contained within it.
./script1.sh data2.txt
Every graphic symbol in the line is displayed verbatim. The backslashes are not interpreted every bit escape characters. They're printed every bit regular characters.
Passing the Line to a Function
We're yet simply echoing the text to the screen. In a real-earth programming scenario, nosotros'd probable exist almost to practice something more than interesting with the line of text. In most cases, it is a skilful programming practice to handle the further processing of the line in another part.
Hither's how we could exercise it. This is "script2.sh."
#!/bin/bash Counter=0 office process_line() { echo "Processing line $Counter : $one " } while IFS= '' read -r LinefromFile || [[ -n " ${LinefromFile} " ]]; do (( Counter ++ )) process_line " $LinefromFile " done < " $1 " We ascertain our Counter variable equally earlier, and and so we ascertain a function called process_line() . The definition of a function must appear before the function is first called in the script.
Our office is going to be passed the newly read line of text in each iteration of the while loop. We can access that value inside the part by using the $1 variable. If in that location were two variables passed to the function, we could access those values using $1 and $two , and and so on for more than variables.
The due westhile loop is mainly the same. At that place is merely one modify inside the body of the loop. The echo line has been replaced past a call to the process_line() function. Notation that you don't need to apply the "()" brackets in the name of the office when you are calling information technology.
The name of the variable holding the line of text, LinefromFile , is wrapped in quotation marks when it is passed to the function. This caters for lines that have spaces in them. Without the quotation marks, the first give-and-take is treated as $1 by the function, the second discussion is considered to exist $2 , so on. Using quotation marks ensures that the entire line of text is handled, birthday, as $1. Note that this is not the same $ane that holds the aforementioned information file passed to the script.
Because Counter has been declared in the main body of the script and not inside a office, it can be referenced inside the process_line() part.
Copy or blazon the script above into an editor and save it with the filename "script2.sh." Make it executable with chmod :
chmod +x script2.sh
Now we can run it and pass in a new information file, "data3.txt." This has a list of the months in it, and 1 line with many words on information technology.
January February March . . October Nov \nMore text "at the end of the line" Dec
Our command is:
./script2.sh data3.txt
The lines are read from the file and passed one by ane to the process_line() office. All the lines are displayed correctly, including the odd ane with the backspace, quotation marks, and multiple words in it.
Building Blocks Are Useful
There'southward a railroad train of thought that says that an idiom must contain something unique to that language. That's non a belief that I subscribe to. What's of import is that information technology makes practiced utilize of the language, is easy to recall, and provides a reliable and robust mode to implement some functionality in your code.
Source: https://www.howtogeek.com/709838/how-to-process-a-file-line-by-line-in-a-linux-bash-script/
0 Response to "How to Read a File Line by Line in Bash Shell Script"
Enregistrer un commentaire