Fork me on GitHub

Project Notes

#002 Process a File/Stream Line-by-Line

:arrow_forward: return to the Catalog

Notes

The bash for..in construct is useful for simple repetitive processing tasks.

When iterating character strings, the behaviour is governed by the internal field separator ($IFS), which determines how Bash recognizes fields, or word boundaries.

$IFS defaults to whitespace (space, tab, and newline).

So when trying to process lines in a file (for example), it will iterate over each word. This may be what you want. But if not, modifying the $IFS allows the behaviour to match what is required.

This is fine for simple text processing. For more complex manipulations, its probably best not to attempt to bend Bash to your will, but instead reach for a more suitable tool (sed, awk, perl, python, ruby etc..).

Line-by-Line Processing Example

See nevermores.sh.

  • it first demonstrates default for..in behaviour
  • then modifies the IFS to scan The Raven line by line and do some processing
$ ./nevermores.sh

Using for..in with default IFS, we get words not lines:
> The
> Raven
> -
> Edgar
> Allan
> Poe

Reset the IFS for newline, then for..in gives us lines:
> Quoth the Raven, "Nevermore."
> With such name as "Nevermore."
> Then the bird said, "Nevermore."
> Of 'Never- nevermore'."
> Meant in croaking "Nevermore."
> She shall press, ah, nevermore!
> Quoth the Raven, "Nevermore."
> Quoth the Raven, "Nevermore."
> Quoth the Raven, "Nevermore."
> Quoth the Raven, "Nevermore."
> Shall be lifted- nevermore!

Credits and References

About LCK#28 Bash
Project Source on GitHub Return to the Project Catalog

This page is a web-friendly rendering of my project notes shared in the LittleCodingKata GitHub repository.

LittleCodingKata is my collection of programming exercises, research and code toys broadly spanning things that relate to programming and software development (languages, frameworks and tools).

These range from the trivial to the complex and serious. Many are inspired by existing work and I'll note credits and references where applicable. The focus is quite scattered, as I variously work on things new and important in the moment, or go back to revisit things from the past.

This is primarily a personal collection for my own edification and learning, but anyone who stumbles by is welcome to borrow, steal or reference the work here. And if you spot errors or issues I'd really appreciate some feedback - create an issue, send me an email or even send a pull-request.