Write code to iterate all files in a directory and append each file's path to a dictionary

All we need is an easy explanation of the problem, so here it is.

I need to achieve the following in my python module.

  1. Get a directory as an argument through the argparse module
  2. Use that directory to iterate through all the files present in that directory.
  3. Later, append only the CSV files to a dictionary and store them there.

I have tried os.walk, but it does not return anything.

I have written code to get a directory as input and then iterate through it for files. But it throws errors.

Below is the code

 import argparse
 import os
 import re
    
 # Accepting arguments for source_directory, my_sql connection details and table name to 
   update the MySQL entries to
    
parser = argparse.ArgumentParser(
        description='Utility to update MySQL tables on remote host')
parser.add_argument('-s', '--source_dir', help='Enter Path For CSV File')
    
args = parser.parse_args()
    
# Empty dictionary to store list CSV files in directory
files_in_dir = {}
    
# Function to iterate through the source_directory and read-only CSV files and append them 
   to files_in_dir dictionary
    
if args.source_dir:
    for file in args.source_dir:
    
        if os.path.abspath(file.name).endswith('.csv'):
            files_in_dir[(os.path.abspath(file.name))] = 0
    
        else:
            print("The file %s is not a .csv file and it will be ignored" % (file.name))
    
print(files_in_dir)

And this is the error when executed through the terminal.

 python3 sourcecheck.py -s /home/thepredator/Desktop/VOIS/csv_files/*
usage: sourcecheck.py [-h] [-s SOURCE_DIR]
sourcecheck.py: error: unrecognized arguments: /home/thepredator/Desktop/VOIS/csv_files/Book1.csv /home/thepredator/Desktop/VOIS/csv_files/Build.py /home/thepredator/Desktop/VOIS/csv_files/car_data.csv /home/thepredator/Desktop/VOIS/csv_files/db.py /home/thepredator/Desktop/VOIS/csv_files/Edited.csv /home/thepredator/Desktop/VOIS/csv_files/Iris.csv /home/thepredator/Desktop/VOIS/csv_files/Lab_Python_for_Data_Analytics.ipynb /home/thepredator/Desktop/VOIS/csv_files/startupNaN.csv /home/thepredator/Desktop/VOIS/csv_files/startupog.csv /home/thepredator/Desktop/VOIS/csv_files/text.csv

I have written the above code by watching different videos. Please help me fix the errors. Also, I’m not a python expert.

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

I changed files_dir = {} line to

files_in_dir = { key:0 for key in os.listdir(args.source_dir) if os.path.abspath(key).endswith(".csv") }

and it seems work.

if-else in for statement version following requests:

for key in os.listdir(args.source_dir):
    if os.path.abspath(key).endswith(".csv"):
        files_in_dir[key] = 0
    else:
        print(f"The file {key} is not a .csv file and it will be ignored")

Method 2

You can use this code`

import os

files = os.listdir("/path_to_directory")    

csv = list(filter(lambda f: f.endswith('.csv'), files))

csv has all the files with .csv extension

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply