Category: Linux

Well THAT'S messy

for file in ../connections_2014-10-7-1*; do conn="-c ../connections_"`echo $file | cut -d "_" -f 2`; types="-t ../neuron_types_"`echo $file | cut -d "_" -f 2`; locs="-l ../locations_"`echo $file | cut -d "_" -f 2`; ./ $conn $types $locs; done

For all the connection files that were generated today, create three variables called “conn”, “types”, and “locs” that have a command line switch path in them generated from a fixed prefix and a cut from the connection file name. Then invoke the script “” with those variables as arguments.

Effectively, the connection, neuron type, and location files are all related by their date, so this makes a single JSON file out of the multiple files. I just didn’t want to run a bunch of times by hand, as that seemed error-prone.

I waste time to not waste time

I’m using a web server on my local machine plus an edited /etc/hosts file to serve up a page that reminds me to get back to work when I should be getting back to work, rather than, say, reading facebook. Yes, I can get around this by clearing my hosts file, but that makes it work to get to the blocked sites, and if I’m going to be doing work, it’s not time-wasting is it?

The hosts file looks like this:

ams@robot-lab7:~/weblock$ cat /etc/hosts localhost.localdomain localhost
#I elided a couple of lines here

# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

That takes care of redirecting a lot of pages to localhost, rather than their real IP addresses. On my computer, I have a little web server, which is launched with the command
sudo webfsd -p 80 -f index.htm -u ams -r /home/ams/weblock/

Webfs, and it’s daemon part webfsd, is a static-file-only http server. In this invocation, it runs as my user after binding to port 80 as root, and serves the file index.htm out of the directory /home/ams/weblock. That file is a very simple HTML file:

<div style=”display: table; height: 100%; width: 100%; background: #fff; _position: relative; overflow: hidden;”>
<div style=”_position: absolute; _top: 50%; display: table-cell; vertical-align: middle; text-align: center;”>
<h1>What should you be doing?</h1>

It just shows the text “What should you be doing?” in the middle of the page. This is also interesting because it shows how to center HTML content in the middle of the page, at least for small content. I’m not sure how well this works with larger or more complex content.

Un-ubuntuing Ubuntu… one of several million, no doubt. Ubuntu is pretty good, but persists in making what I’d regard as irritating desktop usability blunders. One of them is the use of “overlay scrollbars” which are scrollbars that are mostly hidden until you mouse over them. I found this annoying, but whatever, I can deal. It even more or less works, most of the time.

However, it breaks Inkscape. Inkscape’s color picker is a long strip of color patches, which you can click to select stroke and fill colors for drawings. The color picker has a scrollbar. If overlay scrollbars are on, then the entire area of the color picker ends up being the mouse-over area for the overlay scrollbar, so you can’t pick colors.

The fix is to issue the command “gsettings set com.canonical.desktop.interface scrollbar-mode normal” from the command line.

Useful mencoder invocation

mencoder -nosound mf://*.jpg -mf w=1280:h=800:type=jpg:fps=30 -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=2400:mbd=2:keyint=132:v4mv:vqmin=3:lumi_mask=0.07:dark_mask=0.2:mpeg_quant:scplx_mask=0.1:tcplx_mask=0.1:naq -o output_filename.avi

Turns all the JPEG files in the directory you are currently in into a nice quality MPEG-4/AVI file. The width and height in the options after -mf should be changed to match the images. This command line also works for PNG files if you replace both instances of “jpg” with “png”.

Command Line Audio Editing With Sox

For my ritual spoken word software piece, I recruited a bunch of my friends to say the text of the ritual. Each “stanza” of the ritual has a call and a response, so I broke each recording up into individual clips for each call and response. That gave me about 28 files per person, and over 100 clips total.

The different participants all recorded on different hardware, and at different volume levels. I also wasn’t super-precise about trimming the clips, so each file had silence at the beginning.

This left me with two problems: some participants were much softer than others, and some of the clips lagged each other, which made for bad chorus effects.

To trim the clips, I used sox, a Linux tool for manipulating sounds, with the command:

for file in *.wav; do sox $file $file.wav silence 1 0.1 2%

This results in a file named foo.wav.wav for each foo.wav file in the directory, so I cleaned up with:

rename -f “s/.wav././g” *

Note that this scribbles over the originals, so keep backups. I’m glad I did, because 2% turned out to be a little aggressive, and trimmed off the beginning of clips starting with an “ma-” sound, such as “make us a…”. This is likely because the sound faded in slowly, and so got counted as part of the noise rather than the beginning of a sound.

There is useful documentation for the sox silence filter here.

Turning the volume up on the files was done with:

for file in *.wav; do sox $file $file.wav gain -l 8; done

and another pass of rename, as above. Adjust the “8” up or down to suit your needs. Positive numbers make it louder, negative make it quieter.

If you want to preview a sox effect, just replace “sox” in the command with “play”, and leave off the output file. For example,

play myfile.wav gain -l 8

will play myfile.wav with increased gain, but won’t change the file.

We Make Ritual Noise

For a festival that I attend, I’m writing a soundscape in boodler to provide the vocal component for a ritual. Here, I’m going to annotate what I need to do to run Boodler on my laptop, which I’ll have at the festival.

The main thing is that Boodler seems to default to OSS, and I use PulseAudio, so to invoke the ritual, you need to run:

boodler -o pulse –external disturbingrelics com.gizmosmith.disturbingrelics/Example

The -o option tells it to use PulseAudio, –external makes it load from a directory instead of a .boop package for testing purposes, and the rest is the agent to run.

To organize all the sound clips I’m using, I have a boodler package for each person’s reading of the ritual script. The script is in a call and response format, with 14 calls and responses, so each package has 28 audio clips, one each for the call and response. I named all the clips “call_N_…” and “response_N_…” (for N in 1..14) so that the program can figure out the call/response pairs by name.

Each package starts out as a directory with the 28 files and a metadata file in them. For the directory “sage”, I create the package with:

boodle-mgr –import create sage

and then install it with:

boodle-mgr install ./com.gizmosmith.sage.1.0.boop

Converting .MPG to .AVI with Linux programs

I am trying to convert .mpg files from a sony camera into other, smaller versions for distribution. The files are in the mpeg2video codec, with a resolution of 720×480 at 29.97 fps. Normally, I’d use ffmpeg for this, but apparently “This program is only provided for compatibility and will be removed in a future release. Please use avconv instead.” Thanks Ubuntu, I’m sure that won’t screw up a few thousand people’s video conversion scripts.

Anyway, let’s see what man avconv has to say. I want no sound, and I want to start 55 seconds into the film, to cut off most of a boring first minute. Seeking into the video is -ss 00:00:55. The option -an drops the audio. So, avconv -i blue_ball.mpg -an -ss 00:00:55 blue_ball.avi should do it. And, in fact, it does.

Cross-system chat log sync

I created a directory called “pidgin_logs” in my Dropbox folder, backed up my logs, removed the old log directory, and then created a link from where the log directory should have been to the Dropbox directory. Pidgin still starts up fine, and after I do this on all of my systems, they will all log chats to my Dropbox account.

I’m not sure I’m comfortable with the level of trust that this places in Dropbox, but it will be very convenient.

The commands to do it are:

cd ~/Dropbox/
mkdir pidgin_logs
cd ~/.purple/
mv logs logs_backup
mkdir logs
ln -sfn ~/Dropbox/pidgin_logs ~/.purple/logs
mv logs_backup/aim logs/
mv logs_backup/jabber logs/

Chatbot is done

I finished writing the chatbot that I was working on. It consists of a set of scripts to prepare the data, and another script that listens for incoming messages and responds. You can get the code and an overview of how it works here.

Obviously, I’m not publishing my chat logs. Use your own. It is designed to work with Pidgin’s HTML-like format for chat logs, but it could be modified to work on almost any corpus. I really should clean up things like the string cleaning routines, but it worked for class, and that’s what actually matters.

Python Pidgin Dbus Monitor

This is a script that registers a callback with Pidgin’s Dbus interface, and then sends a message in response whenever anyone sends an IM. I’m using it for a chatbot, but it could easily be extended to do things like switch X10 light controllers on and off, get the state of hardware connected to the target computer, and so forth. Pidgin has to be running and configured to use Dbus, but it does that by default.

Created on Nov 18, 2011
Watches pidgin over Dbus and responds to incoming messages.

import dbus
from dbus.mainloop.glib import DBusGMainLoop
import gobject

class StupidResponder():
    def getResponse(self, message):
        #This is where you would do something clever to come up with a response
        return "Insect! I cannot bear your words! They are TOO TINY!"

def got_msg_cb(account, sender, message, conversation, flags):
    purple.PurpleConvImSend(purple.PurpleConvIm(conversation), responder.getResponse(message))

if __name__ == '__main__':

    #load a response generator
    responder = StupidResponder()

    #Connect to pidgin on Dbus
    main_loop = DBusGMainLoop()
    session_bus = dbus.SessionBus(mainloop = main_loop)
    obj = session_bus.get_object("im.pidgin.purple.PurpleService", "/im/pidgin/purple/PurpleObject")
    purple = dbus.Interface(obj, "im.pidgin.purple.PurpleInterface")

    #Add the callback
    session_bus.add_signal_receiver(got_msg_cb, dbus_interface="im.pidgin.purple.PurpleInterface", signal_name="ReceivedImMsg")

    loop = gobject.MainLoop()