How To Load / Export Multiple TXT Files In Python Using Pandas

This article will tell you how to load text file data using the python pandas library. It also tells you how to write pandas DataFrame object back to a text file to save the content. The text file data should be saved in multiple text lines. The text columns in one text line should be separated by a unified character such as ;.

1. How To Load / Export Text File Content In Python Using Pandas.

1. You can use the python pandas module’s read_csv(txt_file_path, sep=sep, header=header) function to load a text file content.
2. You can use the python pandas module’s DataFrame object’s to_csv(target_file, sep=’:’, index = True, header = True, encoding = encoding) function to load a text file content.
3. Because python pandas treat Text files as CSV files, so you can use the above methods.

2. Load / Export TXT Files In Python Using Pandas Examples.

2.1 The Example Text Files.

1. The example text file name is employee_info.txt, and employee_info_1.txt.
2. The employee_info.txt file content.

   Name;Hire Date;Salary
   jerry;2010-01-01;16000
   tom;2011-08-19;6000
   kevin;2009-02-08;13000
   richard;2012-03-19;5000
   john;2017-10-01;16600

3. The employee_info_1.txt file content.

   Name;Hire Date;Salary
   jackie;2015-06-08;28000
   steven;2008-02-01;36000
   jack;2006-09-19;8000
   gary;2018-01-16;19000

2.2 Load Single Text File In Python Pandas Example.

1. load_single_txt_file_by_pandas(txt_file_path, sep, header).

   import pandas as pd
   
   import glob, os
   
   
   '''
   This function load one txt file content with the pandas read_csv() method.
   
   txt_file_path: The text file path.
   
   sep: The text line data separator.
   
   header: The line number in the text file that is used as the column header.
   
   '''
   def load_single_txt_file_by_pandas(txt_file_path, sep, header):
       
       df = pd.read_csv(txt_file_path, sep=sep, header=header)
       
       print(df.head(10))
       
       return df
   
   
   if __name__ == '__main__':
       
       df = load_single_txt_file_by_pandas('./employee_info.txt', ';', 0)

2. When you run the above example code, you will get the below output.

         Name   Hire Date  Salary
   0    jerry  2010-01-01   16000
   1      tom  2011-08-19    6000
   2    kevin  2009-02-08   13000
   3  richard  2012-03-19    5000
   4     john  2017-10-01   16600
   

2.3 Load Multiple Text Files In Python Pandas Example.

1. load_multiple_txt_files_in_folder_by_pandas(folder_path, sep, header).

   '''
   Created on Oct 24, 2021
   
   @author: songzhao
   '''
   
   import pandas as pd
   
   import glob, os
   
   
   
   '''
   This function will load all the txt files data in the provided directory.
   
   folder_path : The txt file saved directory path.
   
   sep : The txt file data line separator.
   
   header : The line number in the txt file that is used as the column header.
   '''
   def load_multiple_txt_files_in_folder_by_pandas(folder_path, sep, header):
       
       # Used to store all the text file content.
       all_data_df = pd.DataFrame([])
       
       # loop all the txt files in the provided directory.
       for txt_file_path in glob.glob(os.path.join(folder_path, '*.txt')):
   
           # If this file exists.
           if(os.path.exists(txt_file_path)):
   
               # Read out all the data in the txt file and return a DataFrame object.
               df = pd.read_csv(txt_file_path, sep=sep, header=header, encoding='utf-8')
               
               # Append the txt file DataFrame object to the global DataFrame variable object.
               all_data_df = all_data_df.append(df)
   
               print('=====', txt_file_path, ' content. ============================================')
   
               # Print out the first 10 lines of the txt file.
               print(df.head(10))
               
               print('=============================================================================')
   
           else:
               print(txt_file_path + " do not exist.")  
       
       print('============ all data in the new dataframe ===============')
   
       # Reset the DataFrame object index to remove the old index and create the new ordered index.
       all_data_df = all_data_df.reset_index(drop=True)
       
       # Print out all txt file's data. 
       print(all_data_df)
       
   
   if __name__ == '__main__':
       
       load_multiple_txt_files_in_folder_by_pandas('./', ';', 0)

2. Below is the above example source code execution output.

   ===== .\employee_info.txt  content. ============================================
         Name   Hire Date  Salary
   0    jerry  2010-01-01   16000
   1      tom  2011-08-19    6000
   2    kevin  2009-02-08   13000
   3  richard  2012-03-19    5000
   4     john  2017-10-01   16600
   =============================================================================
   ===== .\employee_info_1.txt  content. ============================================
        Name   Hire Date  Salary
   0  jackie  2015-06-08   28000
   1  steven  2008-02-01   36000
   2    jack  2006-09-19    8000
   3    gary  2018-01-16   19000
   =============================================================================
   ============ all data df ===============
         Name   Hire Date  Salary
   0    jerry  2010-01-01   16000
   1      tom  2011-08-19    6000
   2    kevin  2009-02-08   13000
   3  richard  2012-03-19    5000
   4     john  2017-10-01   16600
   5   jackie  2015-06-08   28000
   6   steven  2008-02-01   36000
   7     jack  2006-09-19    8000
   8     gary  2018-01-16   19000
   
   

2.4 Export Python Pandas DataFrame Object To Text File Example.

1. export_dataframe_to_txt_file(df_obj, target_file, encoding).

   '''
   Created on Oct 24, 2021
   
   @author: songzhao
   '''
   
   import pandas as pd
   
   import glob, os
   
   
   '''
   This function load one txt file content with the pandas read_csv() method.
   
   txt_file_path: The text file path.
   
   sep: The text line data separator.
   
   header: The line number in the text file that is used as the column header.
   
   '''
   def load_single_txt_file_by_pandas(txt_file_path, sep, header):
       
       df = pd.read_csv(txt_file_path, sep=sep, header=header)
   
       print('\r\n========= ', txt_file_path, ' content ===========')
       
       print(df.head(10))
       
       return df
       
       
   '''
   This function will write the pandas.DataFrame object to the target file with the provided encoding.
   
   df_obj: The pandas DataFrame object.
   
   target_file: The destination file path.
   
   encoding: The text file data encoding character.
   '''
   def export_dataframe_to_txt_file(df_obj, target_file, encoding):
       
       df_obj.to_csv(target_file, sep=':', index = True, header = True, encoding = encoding)
       
       print('\r\n', target_file, ' has been generated.')
       
   
   if __name__ == '__main__':
       
       df = load_single_txt_file_by_pandas('./employee_info.txt', ';', 0)
       
       export_dataframe_to_txt_file( df, target_file = './employee_info_clone.txt', encoding='utf-8')
   
       df = load_single_txt_file_by_pandas('./employee_info_clone.txt', ';', 0)
   

2. When you run the above example source code, you will get the below output, and it will generate the text file employee_info_clone.txt in the folder.

   =========  ./employee_info.txt  content ===========
         Name   Hire Date  Salary
   0    jerry  2010-01-01   16000
   1      tom  2011-08-19    6000
   2    kevin  2009-02-08   13000
   3  richard  2012-03-19    5000
   4     john  2017-10-01   16600
   
   
    ./employee_info_clone.txt  has been generated.
   
   
   =========  ./employee_info_clone.txt  content ===========
         :Name:Hire Date:Salary
   0   0:jerry:2010-01-01:16000
   1      1:tom:2011-08-19:6000
   2   2:kevin:2009-02-08:13000
   3  3:richard:2012-03-19:5000
   4    4:john:2017-10-01:16600