More Webcams, Fewer Problems

I want to have two applications using the same webcam under Linux, in my case an app using Kivy to render the webcam view and let people interact with it, and ROS.

My first hope had been to have a Kivy app that subscribed to a ROS image topic and displayed the resulting image, but I’ve spent two days beating on it and only found a variety of ways to make a python program give me a segfault.

The new plan is to use v4l2loopback to create two virtual cameras, and have them both fed from /dev/video0 (the actual camera).

sudo modprobe v4l2loopback devices=2

That gets me the two devices. The thing is, these devices don’t actually output any video. They’re just fake video devices that a program can write to. There are a lot of instructions on feeding them with ffmpeg. For some reason, my computer says it has ffmpeg installed, but locate can’t locate it. Instead, I’ve used gstreamer to set up /dev/video1 as a video device that ROS’s usb_cam node can handle, and configured the launch file for usb_cam to read /dev/video1

gst-launch v4l2src device=/dev/video0 ! video/x-raw-yuv,width=640,height=480,framerate=15/1 ! v4l2sink device=/dev/video1

That solves half of my problem. The other half is having the second video device get fed, and getting Kivy to read /dev/video2.

gst-launch v4l2src device=/dev/video0 ! video/x-raw-yuv,width=640,height=480,framerate=15/1 ! tee name=t ! v4l2sink device=/dev/video1 t. ! queue ! v4l2sink device=/dev/video2

Adding the tee and queue gives me two video devices playing the same video, both fed by /dev/video0. The usb_cam ROS node can read either one of them. The basic Kivy camera app that I bashed together doesn’t work, though. It seems to default to trying to open /dev/video0, and then failing because the gst-launch invocation is using it.

      id: camera
      play: True

Adding index:2 to my Kivy app’s .kv file gets me a kivy app with my video in it. As long as my ROS nodes are looking at /dev/video1 and my Kivy app is looking at /dev/video2, no one steps on anyone’s toes, and they both can operate at the same time.

Lightlocker does not play well with NVida (or something)

If you have an Ubuntu system with an NVida video card, and after suspending and resuming you get a login screen, but after you log in you get a black screen, and using ctrl + alt + F1 gets you a text console, your problem is probably lightlocker. I’ve seen some sites claim that the problem is fixable with sudo apt-get remove --purge nvidia-* which would be great if you didn’t want video drivers or CUDA. I’d rather hang onto my video drivers, so sudo apt-get remove light-locker makes everything better.

ROS and OpenCV will fite u, m8

I recently wanted to do some computer vision stuff using OpenFace, which is a collection of face-processing computer vision algorithms and tools to use them. It uses OpenCV 3.0.something, which uses, among other things, vtk6, and friends libvtk6-dev and python-vtk6.

Normally, this wouldn’t be a problem, but I use ROS Indigo, as does the lab I work in. ROS Indigo uses some previous version of vtk, and so attempting to install OpenCV 3.0 blows away my ROS install, and makes apt freak out when I try to install it again. The actual error was something like “you have broken held packages”, only I didn’t actually have held packages OR broken packages.

Apt just gives up at this point. Aptitude, on the other hand, proposes removing the offending VTK packages and proceeding with the ROS install. Only time will tell if I’ve trashed my OpenCV install, but if I have, I can just go back to an older OpenCV version.

Download ALL The Music

Given a file containing a list of songs, one per line, in the format “Artist – Song Title”, download the audio of the first youtube video link on a Google search for that song. This is quite useful if you want to the MP3 for every song you ever gave a thumbs up on Pandora. On my computer, this averages about 4 songs a minute.

The Requests API and BeautifulSoup make writing screenscrapers and automating the web really clean and easy.


# Takes a list of titles of songs, in the format "artist - song" and searches for each
# song on google. The first youtube link is passed off to youtube-dl to download it and 
# get the MP3 out. This doesn't have any throttling because (in theory) the conversion step
# takes enough time to provide throttling. 

import requests
import re
from BeautifulSoup import BeautifulSoup
from subprocess import call

def queryConverter(videoURL):
	call(["youtube-dl", "--extract-audio",  "--audio-format", "mp3", videoURL])

def queryGoogle(songTitle):
	reqPreamble = ""
	reqData = {'q':songTitle}
	r = requests.get(reqPreamble, params=reqData)
	if r.status_code != 200:
		print "Failed to issue request to {0}".format(r.url)
		bs = BeautifulSoup(r.text)
		tubelinks = bs.findAll("a", attrs={'href':re.compile("watch")})
		if len(tubelinks) > 0:
			vidUrl ="https[^&]*", tubelinks[0]['href'])
			vidUrl = requests.utils.unquote(
			return vidUrl
			print "No video for {0}".format(songTitle)

if __name__=="__main__":
	with open("./all_pandora_likes", 'r') as inFile:
		for line in inFile:
			videoURL = queryGoogle(line)
			if videoURL is not None:

Further Troubles with TinyRobos

The white version of the TinyRobo board that has the missing ground trace also doesn’t have a proper connection for the pullups on the I2C address lines for the motor drivers. The drivers are still there, and scanning the I2C bus with a Bus Pirate (Amazon) showed me that they were at 0x63 and 0x64 on the I2C bus, rather than where I expected them to be (at 0x66 and 0x68). The difference is consistent with a connection that should have been to Vcc being left open.

I’m not wild about the problem, but it did give me an opportunity to set up and use Pulseview/Sigrok, my cheap clone logic analyzer, and my Bus Pirate, so it’s not a total waste.

For my own future reference, as well as anyone else who’s interested, the way to set up the Bus Pirate on Ubuntu is this:

  1. Plug it into a USB port
  2. Open up a terminal and type screen /dev/buspirate 115200 8N1 , where /dev/buspirate is whatever device your bus pirate ended up on. Mine was /dev/ttyUSB0.
  3. The terminal will go blank. Hit enter, and you should get the “HiZ>” Bus Pirate terminal.

The I2C bus scan is runĀ  by hitting “m” to get the menu, “4” to get I2C mode, “3” to set speed to 100kHz, and then “(1)” to run the scan macro.

The cheapo logic analyzer I got is a USBee & Saleae clone, which I got because I’m bad and should feel bad not rich. It has a switch to determine which device it claims to be. In Saleae mode, Sigrok loads an alternate firmware onto it, so I’m not really sure where that falls in the intellectual property/doing the right thing by small businesses framework, but if you can afford one, get a proper USBee or Saleae. They’re much better built (the Saleae Logics in particular are tanks) and have more and better features.

I’m doing all this stuff in Enschede, at U. Twente. I’ve been hanging out with the people in the HMI group, which “does things with stuff”. They do a lot of work with things like proxemics in interaction, socially aware robots and technology, and so forth. There’s something of a distinction here between technical stuff, which is what I do a lot of, and more abstract work with avatars and such. I’m a better fit with the RaM (Robotics and Mechatronics) group, which builds things like pipe-crawling robots and quadcopters.

Bugs in new boards

I seem to have left a ground connection off the PCB, which causes the 3.3v regulator to not work. I’ve fixed it in the PCB design in the repository, but the DirtyPCBs order link goes to a product that doesn’t work, so I still have to fix that.

Since I’m going to have to do a new version of the PCB anyway, I’ve added a blinky light on one of the IO pins so that I can have an additional channel for debug information.

PDB for n00bs

PDB is the python debugger, which is very handy for debugging scripts. I use it two ways.

If I’m having a problem with the script, I’ll put in the line

import pdb; pdb.set_trace()

just before where the problem occurs. Once the pdb line is hit, I get the interactive debugger and can start stepping through the program and seeing where it blows up, and what variables are getting set to before that happens.

However, I recently found a very handy second way. I was debugging a script with a curses interface, which cleans up when it exits. Unfortunately, that cleanup means that my terminal gets wiped when something crashes, so instead of a stack trace, I just get dumped back to the terminal when something goes wrong, with no information at all left on the screen.

Invoking the script with

python -m pdb ./

gets me the postmortem debugger, so when something goes wrong, the program halts and I get the interactive debugger and some amount of stack trace. It’s messy looking because of curses, but I can at least see what is going on.

New Swarm Controllers

I’ve ordered the second version of the swarm control boards. If you want some, you can get them here, but I advise against doing so until after a post shows up here saying either that they work, or that they’re busted.

In the mean time, I’ve been realizing that the boards are good for all sorts of stupid tricks. For instance, you can control people using galvanic vestibular stimulation, which uses 1-1.5mA at pretty low voltages (More academic version, more hacking). Since the swarm control boards already use a 3.7v lithium cell, additional voltage regulation isn’t needed (if anything, they may be too weak), and PWM can be used to control the current. A resistor in series might also be good, in case of… errors.

The same board could also be connected to a door latch, or magnetic strike, which would let a user connect to a web page (the ESP8266 can serve web pages and act as an AP) and put in a password to open the door. Lockitron appears to be making a business out of selling this, but the mechanics are cheaper.

Given that there’s also an I2C bus on the device, IO expanders, sensors, and other goofiness could be added to make wearables that respond to the environment, smart dust sensors, IoT nodes for home automation, scales that tweet about how much you weigh, etc. IoT is the new black! It’s a floor wax! It’s a dessert topping!




ESP8266, Serial Adapters, and Resets

As detailed in the previous post, I’ve been having some trouble getting the Arduino development environment to automatically reset my ESP8266 board using the DTR and CTS lines of the serial adapter. Part of my problem may still have been the cheap serial adapter, but today I found a new part.

The ESP8266 is extremely sensitive to noise on the CH_PD line, and I was using a 9″ long jumper to connect CH_PD to RTS. I confirmed with my O-scope that RTS was pulsing as it should, but the first pulse threw the ESP8266 into some weird state where it spewed noise on a bunch of pins (GPIO0 seemed to be the worst), and uploading would, naturally, fail.

Switching to 3″ jumpers cleared up the problem and let my Arduino IDE reset the ESP8266 as it should.

I’ve changed the schematic in the Github repo for the project to reflect the new reset wiring, but I still have to add a 5V input connection for charging the battery. Once that’s done, I can design a new PCB.


A Word of Warning

My PhD work (TinyRobo) uses a USB-Serial converter to talk to the ESP-8266 modules in the tiny robots. Normal FTDI cables have a cable that ends in a 0.1″ 6-pin header with this pinout:

  1. Black – Ground
  2. Brown – CTS
  3. Red – VCC
  4. Orange – TX
  5. Yellow – RX
  6. Green – RTS

It turns out that esptool can manipulate the DTR and RTS lines to reset the chip in bootloader mode, which is great for uploading code to it. It also means I can get away with not having any parts on the TinyRobo boards to handle the reset, which is great because it lets me keep the board small. Unfortunately, the FTDI cable I have doesn’t expose the RTS line, so I got a converter module for cheap off Amazon. The particular module I got is this one:

2016-04-15 14.10.13

I added that red wire and cut a trace so that the pins would be:

  1. DTR
  2. RX
  3. TX
  4. VCC
  5. CTS
  6. Ground

So far, so good, but I can’t upload with it. I threw a scope on the lines, and it looks like instead of swinging from VCC to ground like well-behaved TTL serial lines, they swing from VCC to VCC minus some tiny voltage, less than a volt. Adding pull-downs on the lines doesn’t seem to have helped. It could be that the timing is of, but I suspect that somewhere, some cheapskate saved some fraction of a cent on this board, at the expense of it doing the one thing it was supposed to do (YOU HAD ONE JOB).