30 Days of Python: Day 28 Interactive Unix Tools

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 28: Interactive Unix Tools

For today’s project, I went back to my utilities projects and modified them to work well in the interactive python shell. These are the tools that duplicated Unix functionality for windows (cat, head, grep, wc, nl, and tail). The idea was to make using windows more like using the Linux or Unix shell. The tools worked great from the windows command line but sometimes you’d prefer to be in the more powerful python interactive shell. The tools as they were always dumped the output to standard out. It wasn’t possible to chain them together and create pipelines in the python shell.

To remedy this I made each of the tools return a generator. In order to keep the windows command line functionality, I changed the main command to call a wrapped version of the function that sends the generator to standard out. One other feature I added was the ability for each tool to work with either a file object or a file name. Here’s an example of how this pattern was applied to nl the line numbering tool:

def nl_file_name(file_name):
'''Generates numbered lines for a file'''
    with open(file_name) as f_in:
        return (line for line in list(nl_file_in(f_in)))

def nl_file_in(file_in):
    '''Generates numbered lines for an open file'''
    return (str(i)+'\t'+line for i, line in enumerate(file_in))

def nl(file1):
    '''Numbers the lines in the file'''
    if isinstance(file1,str):
        return nl_file_name(file1)
    else:
        return nl_file_in(file1)

def nl_dump(file1):
    '''Numbers the lines in the file'''
    sys.stdout.writelines(nl(file1))

Note that the version that opens the file has to force the generator into a list in order to deal with the file closing. To truly create a pipeline with out worrying about memory issues, it would be best to use the tools with the files already open instead of by file name. Adding the interactive mode to the utilities, leaves me pretty satisfied with my toolset.

 

For those that are interested, here’s my desert island utilities repository: https://github.com/robb07/utilities

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s