How to Effectively Utilize the Python ‘fileinput’ Module to Reads Multiple Files Line by Line

Python’s versatility extends beyond its core functionalities, offering a rich set of modules to simplify complex tasks. Among these, the `fileinput` module stands out as a powerful tool for handling input from both files and streams. In this article, we’ll explore the ins and outs of the `fileinput` module, showcasing its features through practical examples.

1. Understanding the Basics.

  1. The `fileinput` module provides a convenient interface for iterating over lines from multiple input sources, such as files or standard input.
  2. Before diving into examples, let’s understand the basic usage of the module.
    import fileinput
    
    # Create a fileinput object
    with fileinput.input(files=('file1.txt', 'file2.txt')) as finput:
        for line in finput:
            print(line, end='')
  3. In this example, we create a `fileinput` object that reads lines from both ‘file1.txt‘ and ‘file2.txt‘.
  4. The `with` statement ensures proper resource management, automatically closing the input files when the block is exited.
  5. Content of file1.txt.
    Hello I love Python.
    
    Hello World
    
    Yes You can do it
  6. Content of file2.txt.
    Ok
    
    Good
    
    Very Good
  7. Execution output.
    Hello I love Python.
    
    Hello World
    
    Yes You can do itOk
    
    Good
    
    Very Good

2. Iterating Over Lines.

  1. One of the primary use cases of the `fileinput` module is iterating over lines in a file.
  2. Let’s consider a scenario where we want to process each line from a file and perform a specific operation.
    import fileinput
    
    def iterate_over_lines():
        with fileinput.input(files='file1.txt') as finput:
            for line in finput:
                # Perform line-specific operations
                processed_line = line.upper()
                print(processed_line, end='')
    
    
    if __name__ == "__main__":
        iterate_over_lines()
    
  3. Here, each line from ‘file1.txt‘ is converted to uppercase, demonstrating how to integrate the `fileinput` module into a line-processing workflow.
  4. Output.
    HELLO I LOVE PYTHON.
    
    HELLO WORLD
    
    YES YOU CAN DO IT

3. In-Place Editing.

  1. The `fileinput` module provides a powerful feature for in-place file editing, allowing you to modify files directly.
  2. This is particularly useful when you need to replace or update specific content in a file.
    import fileinput
    
    def in_place_editing():
        # Replace 'old_text' with 'new_text' in 'example.txt'
        with fileinput.input(files='file1.txt', inplace=True, backup='.bak') as finput:
            for line in finput:
                print(line.replace('Python', 'Pygame'), end='')
    
        with open('file1.txt', 'rb') as file:
            print('file1.txt : ', file.readlines())
    
        with open('file1.txt.bak', 'rb') as file:
            print('file1.txt.bak : ', file.readlines())
    
    
    if __name__ == "__main__":
        in_place_editing()
    
  3. In this example, the `inplace=True` parameter ensures that changes are written back to the original file (file1.txt).
  4. The `backup=’.bak’` parameter creates a backup of the original file with the specified extension (file1.txt.bak).
  5. After you run the above Python source code, you will find it will create a file file1.txt.bak to save the content of the original file file1.txt.
  6. And the original file1.txt‘s content will be edited as required in the code. Below is the above code output.
    file1.txt :  [b'Hello I love Pygame.\r\n', b'\r\n', b'Hello World\r\n', b'\r\n', b'Yes You can do it']
    file1.txt.bak :  [b'Hello I love Python.\r\n', b'\r\n', b'Hello World\r\n', b'\r\n', b'Yes You can do it']

4. Process Multiple Files.

  1. The `fileinput` module seamlessly handles input from multiple files. This is advantageous when you need to process data spread across several files without explicitly managing each file individually.
    def process_multiple_files():
        # Process lines from 'file1.txt' and 'file2.txt'
        with fileinput.input(files=('file1.txt', 'file2.txt')) as finput:
            for line in finput:
                # Perform operations on lines from multiple files
                processed_line = line.strip()
    
                if len(processed_line) > 0:
                    print(processed_line, end='\r\n')
    
    if __name__ == "__main__":
        process_multiple_files()
  2. Here, lines from both ‘file1.txt‘ and ‘file2.txt‘ are processed within the same loop, streamlining the code and improving maintainability.
  3. Output.
    Hello I love Python.
    Hello World
    Yes You can do it
    Ok
    Good
    Very Good

5. Get Each File Name In The fileinput Object.

  1. To retrieve the name of each file being processed using the `fileinput` module in Python, you can leverage the `fileinput.filename()` function. This function returns the name of the current input file being processed. Below are examples.
    import fileinput
    
    def fileinput_get_each_file_name():
    
        with fileinput.input(files=('file1.txt', 'file2.txt')) as finput:
            
            filename = None
    
            for line in finput:
    
                if filename != finput.filename():
                    filename = finput.filename()
                    print('file : ', filename)
    
                # Process lines while ignoring specific files
                processed_line = line.strip()
                print(processed_line, end='\r\n')
    
    if __name__ == "__main__":
        fileinput_get_each_file_name()
  2. Output.
    file :  file1.txt
    Hello I love Python.
    
    Hello World
    
    Yes You can do it
    file :  file2.txt
    Ok
    
    Good
    
    Very Good
  3. By incorporating the `fileinput.filename()` function into your code, you enhance the clarity and traceability of your file processing operations.

6. Conclusion.

  1. The `fileinput` module is a versatile tool for efficiently handling input from various sources in Python. Whether you’re iterating over lines, performing in-place editing, or processing data from multiple files, the `fileinput` module streamlines these tasks, contributing to cleaner and more maintainable code.
  2. By exploring the examples provided in this guide, you can harness the full potential of the `fileinput` module and enhance your file handling capabilities in Python.
  3. As you integrate these techniques into your projects, you’ll find that the `fileinput` module is a valuable addition to your Python toolkit, offering a convenient and powerful solution for file input processing.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.