Counting Instances of Things

This is something I seem to do a lot. Most recently, I was trying to see how many nodes of each degree there were in a graph, but it could also be used for word counts and such.

#Run through a list and count occurrences of each item,
#store them in a dictionary of items to counts
def bin(list):
    bins = {}
    for item in list:
        if item in bins.keys():
            bins[item] += 1
        else:
            bins[item] = 1
    return bins

The key thing here is not really the little bit of python glue code to do the counting. What is key is to learn to recognize things that you do more than once, and automate them. A folkloric axiom of computer programming is that you need to do something either exactly once, or an unknown number of times. If you have to do it once, then the program should do it once. If it’s the other case, then the program should determine how often to do it, and then do it.

For example, if the program trims the whitespace off the end of a text file, it should either do it once (which allows the user to call it in a shell script loop if they want to process any number of files), or do it to any number of files. The alternative, having it process some arbitrary number of files and then fail, has two problems. First, it adds complexity. The program now has to keep a counter, check it, and fail when it hits a certain value. Second, the program, which previously worked in an infinite number of cases (that is, it would process N files for all N such that N is a positive integer), now has an infinite number of failure cases (fail for all integers greater than M), and has gone from complete success to complete failure, via the most difficult route.

Gallery Plugin Get

[Show as slideshow]

[View with PicLens]

1234►

Photos from the Rhode Island Min Maker Faire a couple of years ago.

Finds

For anyone who has ever wondered where I get my ideas from, I find them in the trash.

I recently found a hollow Styrofoam cube, with one open side. It’s about 14″ on each edge, with walls about 1.5″ thick. I can just barely see overhead lights through it, so I imagine it will light up well with colored LEDs in it.

Furniture building

Most of my projects are on hold until I get a place to work on them. My current lab is about 11′ x 12′, with a little storage room off to the side. It’s not big enough to store my tools and provide a useful workspace, so I’m improving it in two ways.

The first improvement is to cut down on stuff. I own a lot of things that I’m not using, and don’t even have any really clear plans to use, so I’m collecting all that junk and putting it in boxes. Some of it is going to friends who are worse hoarders have clearer plans for it than me. Anything that people don’t take is going to have a chance to get sold at the MIT Flea Market in April, and then is going to get thrown away. Once I’m done getting rid of stuff, I’m going to start tagging things when I use them, and get rid of the stuff that I don’t use for a year. I’m not going to be terribly strict about this, but I am going to give it a shot, and see how much of the stuff I actually want when it comes time to get rid of it.

The second improvement is to make a useful workspace. I used to have a large corner desk, which I used for all sorts of work. Since I didn’t have space for that in my current apartment, I got a smaller desk from a friend. It’s a cute desk, but it isn’t useful. The top has drawers that open over the desk area, so you can only get at the contents of the drawers if you don’t have anything on the desk. That doesn’t happen with desks I own. Aside from that, it’s too small to accommodate tools, a computer, and a project, and since many of my projects involve both a computer and tools, this is a problem.

To replace the useless desk, I’m building a new workbench. It’s a standing desk, with a 5’4″ x 3′ upper surface made of birch plywood. The underside will have some space for storage and my computer. Since the computer desk and the work desk will be consolidated into one workbench, I’ll be able to get rid of both my computer desk and my current workbench, and both the chairs in front of them, and the pads that protect the floors from the chairs. I’ll probably keep at least one of the chairs around, in case I need it in the future, but it won’t be in my lab.

The category of this entry “B Work” refers to work which is intended to accelerate the thing you really want to do, which is your “A work”. “C work” accelerates your acceleration, and so very little of it needs to be done in comparison to the other two.

Bad ideas for NaNoWriMo

I’m not a novelist. I can write, and with sufficient editing, I can even write passable English text, but it’s not really something that I’ve put a lot of effort into, and so not something that I’m good at. My real time investment in writing has been writing software, which is read only by compilers and interpreters, and so what it lacks in excitement, narrative, and plausible dialog, it makes up for in conciseness and precision.

My ideas for National Novel Writing Month (November, for those as don’t know) then, are coding a program that writes novels, and coding a program that takes novels as inputs and generates interactive fiction (text adventure games) from them. Both of these are problems with infinite hair, and are arguably AI-Hard problems (that is, problems whose solution is on par, difficulty-wise, with creating a human-level general-purpose AI). On the other hand, writing a good novel is probably also quite hard.

I think the most reasonable approach for the novel generator would be a recursively-defined novel description language, which selects from tropes and plot stubs, generates characters, and so forth, based on relatively simple rules. The complexity would come from applying the rules over and over, so a simple quest to throw a ring into a volcano grows branches on branches on branches until it is One Damn Thing After Another Until All The Orcs Are Dead. The goal of the program would be to use generative content and emergent behavior to do most of the writing, and leave me to fill out the turns of phrase and details (or generate them a la Dwarf Fortress, which menaces with spikes of ivory). Done badly, this would read like a Mad Lib. Done well, it would read like a Mad Lib filled out by people who don’t say “dongs” every time they are asked for a noun.

Making interactive fiction (IF) out of novels would be substantially harder. The novel parser would have to read English, which is actually quite a trick. English has multiple words for one meaning and multiple meanings for one word, highly flexible structure, and counts on the reader to sort it all out. On top of that, most of the awesome tricks one can pull in English are  more a matter of exploiting shared cultural context with your reader than they are particular sequences of words. If I wrote such a parser in one month, or at all, a lot of linguistics researchers would be out of work.

Assume I went for one tiny part of the problem: identifying the locations in the novel. The same place might be described as “where that party was”, “Joe’s house”, “the darkened house”, and “a pit of iniquity”. Only the events of the novel link them, and so the program would have to determine that these totally different words referred to the same place. The best approximation I could likely come up with is identifying all the things in the novel that sound like places, and then performing some sort of clustering based on what words are mentioned close to mentions of those places. This would likely lead to a bunch of spurious places getting generated, and real places getting overlooked.  There is an entire company, called Metacarta, that did this sort of analysis on much more constrained data sets, and even then it was a difficult problem for a team of people who were likely smarter than me.

However, doing a good job of adapting novels to interactive fiction might not be the best approach. It might be better to get a rough cut of the software together to do anything at all, and gradually improve it until it writes things that are playable curios, rather than detailed simulations of well-loved novels. It wouldn’t be a matter of playing through “A Game of Thrones” so much as it would be “poking around in a demented dreamscape based loosely on ‘A Game of Thrones’”.

This is actually related to another idea that I had, which is sort of a rails shooter based on the consequences of shooting things. You play through the game, riding the rails and shooting enemies of varying levels of craftiness and menace. When you reach the end of the level, you just loop through it again, passing the bodies of everything you killed, and getting another shot at everything you missed. This repeats until you have killed everything in the game world, whereupon you continue to loop, passing through scenes of slaughter as the heroic music fades and is replaced with silence and the buzzing of flies. Perhaps, if you let it run long enough, the dead bodies would rot to skeletons. Now, of course, I’ve spoiled it for you, but I’ll probably never get around to writing it, so at least you’ve had the idea.

Chatbot is done

I finished writing the chatbot that I was working on. It consists of a set of scripts to prepare the data, and another script that listens for incoming messages and responds. You can get the code and an overview of how it works here.

Obviously, I’m not publishing my chat logs. Use your own. It is designed to work with Pidgin’s HTML-like format for chat logs, but it could be modified to work on almost any corpus. I really should clean up things like the string cleaning routines, but it worked for class, and that’s what actually matters.

Python Pidgin Dbus Monitor

This is a script that registers a callback with Pidgin’s Dbus interface, and then sends a message in response whenever anyone sends an IM. I’m using it for a chatbot, but it could easily be extended to do things like switch X10 light controllers on and off, get the state of hardware connected to the target computer, and so forth. Pidgin has to be running and configured to use Dbus, but it does that by default.

'''
Created on Nov 18, 2011
Watches pidgin over Dbus and responds to incoming messages.
'''

import dbus
from dbus.mainloop.glib import DBusGMainLoop
import gobject

class StupidResponder():
    def getResponse(self, message):
        #This is where you would do something clever to come up with a response
        return "Insect! I cannot bear your words! They are TOO TINY!"

def got_msg_cb(account, sender, message, conversation, flags):
    purple.PurpleConvImSend(purple.PurpleConvIm(conversation), responder.getResponse(message))

if __name__ == '__main__':

    #load a response generator
    responder = StupidResponder()

    #Connect to pidgin on Dbus
    main_loop = DBusGMainLoop()
    session_bus = dbus.SessionBus(mainloop = main_loop)
    obj = session_bus.get_object("im.pidgin.purple.PurpleService", "/im/pidgin/purple/PurpleObject")
    purple = dbus.Interface(obj, "im.pidgin.purple.PurpleInterface")

    #Add the callback
    session_bus.add_signal_receiver(got_msg_cb, dbus_interface="im.pidgin.purple.PurpleInterface", signal_name="ReceivedImMsg")

    #Listen
    loop = gobject.MainLoop()
    loop.run()

Some Short Notes

The ToyBrain project has been on hold for a variety of reasons, mostly time and money. I finally have enough money to order the motor driver chips I wanted from Digikey. They are on back order, but should arrive near the end of the month. Once I have those, I’m going to put together a little video of the first boards doing a variety of motor driving tasks. That video will go into a Kickstarter funding round to get the second edition of the boards produced and populated.

At least one of the ToyBrain boards is going to end up hooked to a computer via the serial port at one end, and a vibrating motor at the other end. I’m reviving an old project to add a teledildonics plugin to Pidgin. It will allow a remote user to use commands like /harder and /faster (and of course /softer and /slower) to control the speed of the vibrator. That one may not make it into the Kickstarter video.

I found an interesting post about laser power ratings recently. It covers the relationship between PWM and laser output power, which is going to be useful for the power supply that I’m building for my laser. Once I build that power supply, I’ll be in the rather interesting position of having designed a cutting laser power supply that can be built from easy-to-obtain materials. Hopefully, that will knock the price down enough that more people can do DIY CNC laser builds. I may also make that PCB available as a kit, so people can build their own.

I also looked up TEA Laser plans again, and started wondering about making a dye laser. The TEA laser emits in the UV range. so it could be used to pump a UV-reactive dye. Vitamin b12 (in energy shot drinks) and tonic water both are UV reactive, so it may be possible to make a yellow or blue laser using a dye that is drinkable. Normally, laser dye is a toxic dye in a toxic solvent, so this would be pretty neat for the home experimenter.

Behold!

I have HEALED this man’s WOUNDED STEREO!

One of the users of a mailing list that I’m on had a nice stereo receiver with one bad channel. Stereo has two channels, and while one worked, the other simply had no output. I opened it up and found that the signal lines from the amplifier run through a relay to the rear panel connectors. I can think of a couple of reasons for this, but the main one is that when you power it up, there is a period where the circuitry of the receiver stabilizes and effectively “boots up”. During that period, it might put some sort of really loud signal out over the speakers (and blow them inside out) unless they are disconnected. Once everything settles, it is safe to connect the speakers. The other possibility is that if you short the speaker connection, the receiver can detect the excessive current flow and disconnect the speaker hookups.

Whatever the reason for them, one of the relays was DPST, with one pole for each stereo channel, and one set of the poles didn’t close when the system powered up. I figured this out by plugging in an audio source, turning the receiver on, and then tracing back from the connector until I found an audio signal. There was nothing at the connector, nothing where the connector joined the PCB, nothing at one pole of the relay, music at the pole it was supposed to be connected to.

The fix was to remove the board with the relay on it, pop off the case of the relay, clean the contacts, push them a little closer together, and put it all back together. If the relay had really been shot, it was a pretty common size, and would have been easy to replace. Overall, the receiver was well-designed for repair, and the service manual was available online. I will give Onkyo props for using one size of normal philips-head screw throughout their case, but did they really need 41 of them?

Notacon Talk on Brain Hacking

This is the talk I gave at Notacon in 2008. It’s kind of goofy, but provides a broad overview of wireheading/neurohacking technologies.