30 Days of Python: Day 18 Breakout

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 18: Breakout

I made the game breakout for today’s project. Breakout is another classic game where you use a paddle to bounce a ball up at a wall of bricks, breaking the bricks one at a time. If the paddle is moving when the ball bounces off of it, some of the paddle’s velocity gets imparted onto the ball. That means you can speed up the ball or slow it down if you are careful. More often than not though, the ball gets going too fast and you lose it. Another fun feature I added was an extra multiplier for the score when you get more than block before the ball hits the paddle again. If you break into the space above the bricks you can get very long runs going. Here’s what the game looks like:

Breakout

Since this is the fifth game in my games repository, I tried to focus on putting any new features into the sprite class instead of writing one off code for breakout. In fact, I didn’t make any new classes for this game. All the pieces are vanilla sprites with some game functions to manage the objects. To the sprite class I added the ability to set the draw and update methods from some standard methods included in the sprite module. The default draw method is a rectangle but now there’s a function for making a sprite be a circle, which the ball uses. The update method is used to move the objects around on the screen with some velocity. In addition to the default of objects moving on to infinity forever, there are three update methods that each handle the walls in different ways:

  1. update_bounce: sprites bounce off of the world’s walls
  2. update_toroid: sprites pass through and wrap around to the opposite side to come back onto the screen
  3. update_stay_in_world: sprites stop moving at a wall or slide along the wall if they were moving at a diagonal

The ball from breakout, the snake from snake, and the paddle from breakout are examples of each of these update methods respectively. Tetris could use the update_stay_in_world method but I haven’t changed that over yet because the nested blocks make it a bit more complicated. Overall, I’d say I’m starting to see how to make python code that reuses old code and doesn’t necessitate excessive new code for every project.

For those who are interested here’s my github repo for the games project: https://github.com/robb07/python_games

30 Days of Python: Day 17 Cat for Desert Island Coding VI

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 17: Cat for Desert Island Coding VI

For today’s project, I made the unix tool cat (short for concatenate, not actually a cat). This is another project inspired by David Beazly’s PyCon 2014 talk (minute 22) where he explains how he built up the software he needed using just python. Over the past few days of projects, I’ve filled out the toolset that I’ve been making quite nicely with headtailgrepwc, and nl. Also, I just got piping to work with the tools. Pipes allow one tool’s output to pass directly to the next tool which can make the combination of tools quite powerful. To start a pipe though it really is necessary to have cat which opens a file and sends it to standard output.

Most tools can open a file and operate on it, but what makes cat special is that it can open multiple files and send each one to standard output, one a time. That way you can turn four files into one, and send them all through the same processing chain. Here’s how you’d combine two files, find the lines with the word ‘python’ and then see the first five lines, all with my python tools:


cat.py C:\Python27\README.txt C:\Python27\LICENSE.txt | grep.py python | head.py -n=5

Another cool feature that I didn’t know about before I implemented my version was that cat can combine the standard input with a series of files by using ‘-‘ as a stand in for standard input. For instance, to add a header to a file you could call cat like this:

my_data.csv
-----------
1,blue,M
2,red,L
3,green,S

echo id,color,size | cat.py - my_data.csv > my_data_w_headers.csv

my_data_w_headers.csv
---------------------
id,color,size
1,blue,M
2,red,L
3,green,S

Adding cat to the mix really rounded out the toolset. There are a few more that I might add eventually but these really are the cornerstone to a lot of text manipulation.

For those that are interested, here’s my desert island utilities repository: https://github.com/robb07/utilities

30 Days of Python: Day 16 Piping on Windows for Desert Island Coding V

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 16: Piping on Windows for Desert Island Coding V

Today’s project was to make pipes work with the various tools that I made for my utilities package. This is another project inspired by David Beazly’s PyCon 2014 talk (minute 22) where he explains how he built up the software he needed using just python. I’ve already built head, tail, grep, wc, and nl in this way, but until today they ran separately on whatever file was supplied as an argument.

If you are familiar with the linux/unix toolset you’ll know that these tools can read from the standard input instead of from a file. This turns out to be a really powerful feature when combined with the Unix pipe (|) which simply reads in the standard output from one program and sends it to standard input for the next program.

It looks like this: head my_file.txt | grep hi which would take then first 10 lines of a file and match them to the pattern “hi.”

This simple idea is at the root of the Unix Philosophy, which came out of the development of Unix. Essentially, the idea is that instead of making huge powerful software tools for every project, build small simple tools that each do one thing really well and then make them work together. When you first look at these tools they seem so basic that its hard to imagine they are of any use, but because they are all designed around the idea of reading in text, manipulating it, and writing it out they can do hugely powerful things when stitched together via piping. I used to munge around text in excel defining each operation and then dragging the formula down each row. I always bemoaned how terrible the formula syntax was and thought there should be a better way. Luckily, there is – the Unix tool set. At first, it seemed like magic; there was always another tool to tack on to the pipe to do the next operation.  But it isn’t magic; engineers designed it 30 years ago and they did it so well that it’s still fundamental today.

For this project, I’m working on Windows and building my linux toolbox up from scratch with just Python. How could I get the tools I made to work together? Happily, I discovered that Windows does support piping. I’m not sure why. The only native command that I can find to use with it is echo. But it’s there so I didn’t have to reinvent that wheel.

To modify the tools to read from standard input when no file is given I made the positional argument optional (nargs='?') and the default value sys.stdin. I also had to set the type to be a read file so I had to move some of the code around to support that. A good example of the new version is head:


def head(file_in, N=10):
    '''Prints the first lines of a file'''
    sys.stdout.writelines(islice(file_in, N))

def main():
    ''' Process command line options '''

    # Setup argument parser
    parser = argparse.ArgumentParser(description='Prints the first lines of a file')
    parser.add_argument('FILE', nargs='?', help='The file to print lines from', type=argparse.FileType('r'), default=sys.stdin)
    parser.add_argument('-n','--lines',help='The number of lines to print',default=10,type=int)

    # Process arguments
    args = parser.parse_args()

    # Unpack the arguments
    file_in = args.FILE
    N_lines = args.lines

    #file_in is an open file
    head(file_in, N_lines)
    file_in.close()

Applying this technique to all of the tools allows for piping from one program to the next. For instance, I could use grep to find all the lines with the word ‘python’ in a file, and then send that to wc to count the number of lines, or I can number the lines with nl so I see which lines matched my pattern (or any other combination of the tools I’ve made). The key is I don’t need to design new software for every use. I can construct it from the basic building blocks I already made.

Here’s a few more examples:

piping

The key idea behind the Unix Philosophy is that if you make small simple tools, you can then use those tools over and over. That saves on work, on time, and it just makes doing this sort of thing more enjoyable. With what is growing into a functional toolset – now including piping – Windows is becoming much more flexible.

My next project is probably going to be cat, a simple tool that outputs a file to the screen but is often the first command in a pipe.

For those that are interested, here’s my desert island utilities repository: https://github.com/robb07/utilities

 

30 Days of Python: Day 15 Grep Pattern Matching for Desert Island Coding IV

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 15: Grep Pattern Matching for Desert Island Coding IV

Today’s project was to build the grep tool for my utilities package. This is another project inspired by David Beazly’s PyCon 2014 talk (minute 22) where he explained a time when he built up the software he needed using just python in a locked vault. Grep is a classic linux tool that let’s you find particular lines in a file. For instance, calling grep with a pattern of “error” would allow you to pull out all the error lines in a log file.

My version can match a pattern exactly, ignore case, invert the match, or use the Python regular expression package, re. The classic Unix grep has a Perl regular expression mode but since I don’t know Perl I figure I’d just change that option to Python. To make this tool I relied heavily on generator expressions:


def grep(...)
    #inside grep
    ...
    lines_and_matches = ((line, re.search(pattern, line, flags=flags) is not None) for line in f_in)
    ...
    if not invert_match:
        lines = (line for line, matched in lines_and_matches if matched)
    else:
        lines = (line for line, matched in lines_and_matches if not matched)
    ...

I’m a big fan of how you can compose them which allowed me to pass one generator to another to get all of the various modes implemented without it becoming too unwieldy. This is a very handy feature to have because a list comprehension would read in the file at each point. Just to give a picture here’s a screen shot of the help, which shows all the implemented options:

grep

In linux I find grep to be quite useful for pulling out particular result lines or constructing more elaborate function calls based on the results of previous commands. All of that relies heavily on the ability to pipe one command into the next. Windows supports pipes but my implementation of grep and my other tools rely on opening a file so that might be the next project.

For those that are interested, here’s my desert island utilities repository: https://github.com/robb07/utilities

 

30 Days of Python: Day 14 Properties, A Python Idiom

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 14: Properties, A Python Idiom

Today’s project was to learn how to use properties in Python. I put this project in my queue back when I was working on Tetris and Snake. Because I had learned to write ‘set’ and ‘get’ methods from dabbling in Java, I found myself habitually using them all over the place to handle the position, velocity, and rotation of the tetris piece, blocks, sprites, and snake segments. It was really tedious, and I knew that Python was supposed to better than this.

Through a quick Google search, I found PJ Eby’s Python is not Java:

Pretend that Python is a magic wand that will miraculously do whatever you want without you needing to lifting a finger. Ask, “how does Python already solve my problem?” and “What Python language feature most resembles my problem?” You will be absolutely astonished at how often it happens that thing you need is already there in some form. In fact, this phenomenon is so common, even among experienced Python programmers, that the Python community has a name for it. We call it “Guido’s time machine”, because sometimes it seems as though that’s the only way he could’ve known what we needed, before we knew it ourselves.

This was a promising start, especially since he also said, “Getters and setters are evil. Evil, evil, I say!”

Instead of translating your coding behavior from Java to Python directly, he suggests you take a step back and learn the features of Python that allow you to truly use the language – not fight it. These features of Python are what are colloquially known as “Python Idioms.” Features such as  list comprehensions, properties, and many others are distinct in Python. They don’t directly translate. I knew roughly what an idiom was, but since it kept showing up in every Python blog post and question on Stack Overflow (the Q&A forum), I thought it best to get a solid definition from wikipedia:

 An idiom is a combination of words that have a figurative meaning owing to its common usage.

Which is simple enough, but my first reaction was: why would you want to fill a language that strives for clarity with idioms? Wouldn’t you want clear, simple words with no figurative meaning? But as I thought about this more I realized that idioms are what actually make a language clear, even if not at the word by word level. When you literally translate one language to another, say French to English, you end up with all sorts of awkwardness: “s’il vous plait” becomes “if it pleases you.” While that works, what the speaker really means is simply “please.” In English you can say “Fred kicked the bucket” which means “Fred died” but those two aren’t quite interchangeable. There is a subtle difference. The former accomplishes more. It removes some of the sadness and it adds some separation to the event, softening the blow. An idiom carries extra meaning beyond what is indicated by the individual words – it’s meaning is greater than the sum of its parts.

When switching from Java to Python you can’t translate word for word. You have to translate at the figurative level; don’t literally translate, idiomatically translate. Here’s an example of a the property idiom (the last few lines):

class Tetroid(sprite.Sprite):
    ...
    def set_pos(self, pos):
        '''Sets the position of the tetroid'''
        self._pos = pos
        self.update_blocks()

    def get_pos(self):
        '''Gets the position of the tetroid'''
    return self._pos

    pos = property(get_pos, set_pos)

    ...

my_tetroid = Tetroid(old_pos,...)
#blocks are updated without calling a set method:
my_tetroid.pos = new_pos

I highly recommend Ryan Tomayko’s take on the specific getter/setter problem in this follow up to PJ Eby as a beginner’s introduction to using properties instead of ‘sets’ and ‘gets’ littering your code. Having started my journey with the frustration of the tedium of ‘sets’ and ‘gets’, I found this line particularly inspiring:

Programming languages are for humans, not machines. If you have code that looks like it doesn’t do anything useful, is hard to read, or seems tedious, then chances are good that Python has some language feature that will let you remove it.

Now for my particular usage of properties:

  • I deleted useless straight pass through sets/gets
  • Sprite has a position property that ensures it has its own copy of the list when the position is set
  • Sprite has rotation and rotation matrix properties, making it easy to calculate offset points that rotate relative to the sprite for drawing
  • Tetroid inherits from Sprite

That last one was really cool to get working and is my best example of the property idiom being more than just sets and gets. Tetroid is a sprite that has a list of four blocks which are also sprites. It was its own object type that had special rotation features that duplicated Sprite features but handled updating its four blocks. Now to update the Tetroid position you can just set its position with a new value. The property takes care of finding the set method which then updates all the blocks and the whole thing moves. The same thing works for rotation. Worrying about updating all the parts in many places is replaced with updating the whole and letting it take care of the rest.

Now I will be the first to admit that my usage of properties is still not idiomatic. I know decorators can do very cool things but I just haven’t learned how to use them yet. From PJ Eby’s post, it seems like there’s a way to use functions of functions to get the input handling that I want with even less duplication. So there is more work to be done, but the work has already paid off. The latest version of Tetris has a proper preview pane and it was easy to write because I could just set the Tetroid’s position and move on.

Tetris with Preview

Learning how to use properties in Python – and seeing that programming by idiom is more that just a buzzword- has made me excited to use the Python magic wand. I look forward to taking many more trips in Guido’s time machine.

 

For those who are interested here’s my github repo for the games project: https://github.com/robb07/python_games

30 Days of Python: Day 13 World Builder

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 13: World Builder

I’ve been off for a few days of vacation but I’m back with a project today. Toady’s project is a world builder that allows me to place blocks with the cursor so I can make worlds easily without having to mentally map it all out. The world builder tool creates a new world object and then adds blocks to it where ever I click. It doesn’t load from or save to a file yet but I figured out how to use __repr__ to print out the objects in a way that I should be able to load from a saved file eventually. Here’s a sample world that I built just while I was testing:

world builder

For those who are interested here’s my github repo for the games project: https://github.com/robb07/python_games

 

30 Days of Python: Day 12 HTML Grabber

I’m making a small project every day in python for the next 30 days (minus some vacation days). I’m hoping to learn many new packages and  make a wide variety of projects, including games, computer tools, machine learning, and maybe some science. It should be a good variety and I think it will be a lot of fun.

Day 12: HTML Grabber

Today’s project is a bit different from the previous from my earlier projects. I have a plan to look at the content of the front pages of various websites to see what words get used to convince people to click on links. But to do that I need lots of front pages from each site. So I made a little tool that can run in the background and grab the front page of each site once an hour.

I’m using the time package to get a time stamp for each file I save as well as for the delay function in a never ending while loop. I’m using urllib2 to pull down the webpage. I know that there are other packages setup for this in a more sophisticated way but I wanted to build a bare bones version to see if I could get it running. So far so good, although I am nervous about getting blocked by the websites. I tested with a higher frequency than I will need to do and one started throwing an error. Fortunately, I have a try/except block to keep the program going in case of an error.