July 30th, 2011 by Niki

Bit.ly held another Hackathon last night, with the intent of playing around with the data available from 1.USA.gov’s developer resources. We were given JSON data representing which government URLs were being visiting, along with user-agents, location, time and other relevant information.

At first I was more interesting in spending my time on a completely unrelated project – writing OCaml bindings to libsmf, a Standard MIDI File library. My efforts can be found on the OCaml smf library page or on my github.

Soon, however, I started to look around at other people’s projects and decided I should participate. Since I was working on a MIDI library, I decided that the perfect project for me would be to translate the data into MIDI. Using the Yojson library to parse the data and OCaml Portmidi (another MIDI library I work on) to generate MIDI events, I created a simple program that hashed the URL to determine which note to play, and hashed the user-agent to determine how long to play the note.

In order to better share this with other people, I went back to working on the smf library and added in the functionality to allow writing MIDI data to a file. Once that was complete, I added that into my Bit.ly project and created a MIDI file. Using timidity I converted it to an mp3 so I could share it with the internet!

github project page


Download the mp3
Download the MIDI

3rd Global Game Jam

February 2nd, 2011 by Niki

Last weekend I attended the 3rd annual Global Game Jam, hosted at NYU. The Global Game Jam is a 48 hour video game development competition – you are given a theme and you have two days to create a game based on that theme!

I arrived a little bit late to the game jam this year, but when I got there people were still forming teams. A friend of mine informed me that this year’s theme was “Extinction.” I mingled for awhile, chatting with old friends and discussing ideas. Groups had already started to form, but none of them had ideas that interested me or they already had enough programmers. My friend (and former co-worker) Yury Pavlotsky was also looking for a team with little success. We discussed our ideas and when we realized we both wanted to create an eco-system simulation we decided to team up. Since we are both programmers we spent a few minutes wandering around looking for an artist to join our team. With only a few “I have a team, but if I have time…” responses we decided to just start coding and worry about the art later.

Working with Yury could not have been easier. We both immediately agreed that we would use Microsoft XNA. We spent only about a half an hour discussing the design elements with almost no conflicts. While most teams were busy arguing over what kind of game they should make, what platform they should target, etc., we were already coding. Our idea was pretty simple – we’d have a grid of cells, and each cell could either be: empty, a plant, an herbivore or a carnivore. The plants would grow and the animals would eat, reproduce and die. The player’s job was to keep the ecosystem in balance.

Each plant/animal had its own stats – how long it lived, how often it reproduced, how much air it breathed, etc. If the air became too CO2 heavy the animals would die, conversely if it became too saturated with oxygen the plants would die. However we found this was extremely difficult to balance. Minor changes would result in huge differences with typical behavior being either everything dying too quickly, or the ecosystem rapidly bouncing back and forth between plants and animals. We ended up taking out the air balance (and later, putting it back it, but only for the plants).

The plants were pretty simple – they would reproduce asexually if there was an empty cell next to it. The animals, however, could move around and needed food to survive. They also needed a mate to reproduce. At first their behavior was totally randomized and based solely on the cells in their immediate area. This stupid behavior would result in the animals dying from starvation or simple not reproducing quickly enough. To fix this I decided that the animals should seek out food if they were starving and a mate if they were at the peak of their reproductive cycle. This involved a simple breadth-first search. What was really interesting, however, was the emergent behavior it caused. Each animal acted independently, and yet we would see flocking/swarming behavior! Groups of animals had the tendency to get hungry together and would all seek out food at the same time.

The search algorithm would cause significant slow down when there were a lot of animals and little food. After some simple optimizations (removing almost all memory allocations in the search) the speed improved but not enough. I eventually added a limit to the number of vertexes it would search. This too led to some interesting behavior – animals could only “see” food that was within a certain distance. Since the animals would tend to group together (due to mating) and would flock together, it would often result in the ecosystem rotating – the herbivores would eat all the plants on one side while the plants grew on the other and would start to “chase” the plants. The carnivores would follow the herbivores in the same manner.

At this point the programming was almost entirely done and we could finally think about art, gameplay and a name. Yury had made some placeholder “programmer” art earlier and we ended up loving the aesthetic. Some of our programmer friends offered to make new art for us but they agreed that the current style was perfect and didn’t really need anything from them. With the art and programming out of the way, we spent the last few hours tweaking the stats for each organism, trying to get it so that it was impossible to create a stable eco-system of just plants and herbivores, and getting it so that the eco-system was just barely unstable with all three (otherwise the game wouldn’t end!). We also needed a name but had trouble coming up with something good. I knew I wanted it to sound kind of “old-school” to match the aesthetic and suggested that our title be an acronym that extinction should be spelled “xtinction”. It was at that point that things went downhill and we took the Juvenile approach: Simulated Environment Xtinction Simulator.

We were done! Now it was time to relax and wait for the demos. There were almost 20 games produced at NYU and each team got a few minutes to show off what they made in front of everyone else. There were a lot of really great games! My favorite was “Ned, you really suck the life out of a room” made by a few friends of mine. Other impressive games were “VS” (also made by some friends), “Goldbloom” (about the death of print media) and “Thawed” (a text adventure about polar bears). After the presentations each team got to vote for the following categories: best overall, most beautiful, best use of theme, wild card and most potential. NYU also had a panel of judges that would choose one game as their favorite. “Ned” ended up taking best overall, most potential and the judges choice. Our game won best use of theme and got an honorary mention from the judges!

The game and all the code and assets can be downloaded from the global game jam website.

Buffered text in dvtm

September 10th, 2010 by Niki

I love dvtm. I do a lot of development over ssh these days and having an application handle multiple terminals is practically a necessity. At first I was using gnu screen, however I found it a little cumbersome to use for my purposes. gnu screen doesn’t have many layout options – you are either looking at one terminal or several terminals arranged in a single stack. Switching between these is annoying as well. dvtm on the other hand has several different layouts and makes it easy to switch between them.

Gnu screen, however, does have several features that dvtm does not. For example, with screen you can detach from the controlling terminal and reattach later. Fortunately dtach takes care of that and it’s easy to run dvtm inside dtach. Another great feature of gnu screen is that it reflows text – when you resize the terminal screen will automatically word wrap the text. dvtm doesn’t do this and I haven’t found any unix shell that does this either. What’s worse is that dvtm will cut off the text when a terminal is made smaller and that text is gone – even if you later restore that terminal to its original size.

Reflowing is a fairly non-trivial thing to do so I opted for the next best thing – making sure that dvtm doesn’t permanently cut off the text. This actually turned out to be fairly easy – dvtm keeps a character array for each row in each terminal. This array is realloced every time a terminal is resized. When a terminal decreases in size, the arrays are realloced so they are smaller. I simply changed the behavior so that they arrays are only realloced when a terminal increases in size such that it’s larger than the previous largest. This means that dvtm’s memory usage is potentially greater, but it also means that it makes fewer memory allocations.

You can find my patch on the dvtm project page.

dwm and dvtm patches

June 2nd, 2010 by Niki

A new version of dwm has been released so I have updated the Fibonacci spiral patch. I’m still working on the movestack patch. The updated dwm patch can be found here. I have also created a Fibonacci layout patch for dvtm – the dynamic virtual terminal manager. The dvtm patch can be found here.

Signing Android applications

May 15th, 2010 by Niki

In order to install an Android Application onto the emulator or the phone you must first cryptographically sign it. When you do a debug build (using “ant debug”) ant will automatically sign the binary with an auto-generated debug key.

This key is set to expire 365 days after it was auto-generated (the first time you did a debug build). If a debug build fails due to an expired key, you simply need to delete the current debug key and a new one will be auto-generated. To do this, navigate to where the debug key is located and delete it. On Linux and Mac OS X this is ~/.android and the key is the file “debug.keystore”.

Signing your application with a debug key is fine for testing purposes but for release the application should be signed with a unique key that you have generated just for that purpose. If you plan on release the application on the Android Market then the key should also contain correct identifying information (your name or company) and should expire no earlier than October 22, 2033.

To generate a key you need to have the keytool application that come with Java. You need to specify the key-store (the file name), the alias for the key, which algorithm to use (we want RSA) and how long it should be valid for. Let’s create a key stored in the file “release.keystore” with the alias “release” that will be valid for 10,000 days:

niki@redblacktree:~/.android$ keytool -genkey -v -key-store release.keystore -alias release -keyalg RSA -validity 10000

Keytool will then ask you a bunch of questions:

Enter keystore password:
Re-enter new password:
What is your first and last name?
[Unknown]: Niki Yoshiuchi
What is the name of your organizational unit?
[Unknown]: aplusbi
What is the name of your organization?
[Unknown]: aplusbi
What is the name of your City or Locality?
[Unknown]: New York
What is the name of your State or Province?
[Unknown]: NY
What is the two-letter country code for this unit?
[Unknown]: US
Is CN=Niki Yoshiuchi, OU=aplusbi, O=aplusbi, L=New York, ST=NY, C=US correct?
[no]: y

Generating 1,024 bit RSA key pair and self-signed certificate (SHA1withRSA) with a validity of 10,000 days
for: CN=Niki Yoshiuchi, OU=aplusbi, O=aplusbi, L=New York, ST=NY, C=US
Enter key password for
(RETURN if same as keystore password):
[Storing release.keystore]

And you’re done! You’ll notice that I ran keytool out of the ~/.android directory so that release.keystore is located in the same place as debug.keystore.

Now onto signing applications. Let’s go back to our Hello Android project and do a release build:

niki@redblacktree:~/projects/android/hello$ ant release

Once this has finished there should be a new apk located in the bin directory called “HelloAndroid-unsigned.apk”. We need to sign it with the key we created earlier using jarsigner:

niki@redblacktree:~/projects/android/hello$ jarsigner -keystore ~/.android/release.keystore HelloAndroid-unsigned.apk release

And now “HelloAndroid-unsigned.apk” has been signed. It can now be installed using adb.

Creating an Android project from the Command Line

May 14th, 2010 by Niki

Android’s developer guide has a section on Developing in other IDEs for those who don’t want to use Eclipse. However the documentation leaves out a few details and all of the examples use Eclipse with perhaps a token paragraph discussing the CLI. The rest of this post will assume that you have already installed the SDK and properly configured it (added the SDK to your path and installed the appropriate SDK components).

The first thing to do is to figure out which version of Android you want to target. Version 1.6 (aka – “Donut”) is a pretty good lowest-common-denominator. Using the Android tool we will figure out the ID for 1.6:

niki@redblacktree:~$ android list targets

Which will give you a list of all the targets available (this depends on which SDK components you have added). My output looks like this:

Available Android targets:
id: 1 or "android-3"
Name: Android 1.5
Type: Platform
API level: 3
Revision: 1
Skins: QVGA-L, QVGA-P, HVGA (default), HVGA-P, HVGA-L
id: 2 or "android-4"
Name: Android 1.6
Type: Platform
API level: 4
Revision: 1
Skins: WVGA854, QVGA, HVGA (default), WVGA800
id: 3 or "android-7"
Name: Android 2.1
Type: Platform
API level: 7
Revision: 1
Skins: WVGA854, WQVGA400, QVGA, HVGA (default), WVGA800, WQVGA432

The target ID for version 1.6 is 2. Knowing that, let’s set up an Android Virtual Device (AVD) for our emulator using the android tool, specifying the target with the –target option and giving it a name with the –name option. We will use the default hardware configuration:

niki@redblacktree:~$ android create avd --target 2 --name donut
Android 1.6 is a basic Android platform.
Do you wish to create a custom hardware profile [no]
Created AVD 'test' based on Android 1.6, with the following hardware config:

Finally, let’s setup a project. Again well use the android tool, this time to create a new project. I’m going to create the project in the directory ~/projects/android/hello:

niki@redblacktree:~/projects/android/hello$ android create project --name HelloAndroid --activity HelloAndroid --path ./ --package com.examples.helloandroid --target 2

This will create a new project in the current directory (–path ./) with the name “HelloAndroid” (–name HelloAndroid). It will be targeted towards Android 1.6 (–target 2). It will create a Java source file containing an Activity “HelloAndroid” (–activity HelloAndroid) and will be in the namespace “com.examples.helloandroid” (–package com.examples.helloandroid).

We can now use Apache Ant to build our project:

niki@redblacktree:~/projects/android/hello$ ant debug

This will build a debug version of our project and place the “HelloAndroid-debug.apk” in the bin directory. This application has been signed using a default debug key and is ready to be installed on a device. To do that, let’s fire up the emulator, using the virtual device we created earlier:

niki@redblacktree:~$ emulator -avd donut

Once the emulator has finished booting, we can get a list of attached devices using the Android Debug Bridge or adb:

niki@redblacktree:~/projects/android/hello$ adb devices
List of devices attached
emulator-5554 device

If there is only one device attached, then you can install the application using ant:

niki@redblacktree:~/projects/android/hello$ ant install

Which will automatically remove any previous versions installed on the device and install the current build. Alternatively you can use adb to install the application:

niki@redblacktree:~/projects/android/hello$ adb install bin/HelloAndroid-debug.apk

This will fail, however, if you have already installed a previous version of the application on the device. If that is the case, you need to uninstall it first:

niki@redblacktree:~/projects/android/hello$ adb uninstall com.examples.helloandroid

Note that you must specify the application by the name of its package.

Finally if you have more than one device attached, (say, a phone and an emulator) then you must specify which device (using the name returned from adb devices) using -s:

niki@redblacktree:~/projects/android/hello$ adb -s emulator-5554 install bin/HelloAndroid-debug.apk

Now you can just navigate to the application on the emulator or on your phone and run it! You’ll notice by default it says “Hello World, HelloAndroid!”

That’s the basics of setting up, building and running an application using the CLI instead of eclipse. In the next post I’ll go over release builds and signing your applications.

Android Development without Eclipse

May 13th, 2010 by Niki

I have owned an Android-based phone (the G1) for awhile now and have been meaning to develop applications for it since day one. I have finally gotten around to learning how. So far it hasn’t been easy for me: I don’t know Java and I don’t use Eclipse which are the primary development tools for Android.

Fortunately Google released the Android NDK, or Native Development Kit which can be used to program applications in C or C++ and be compiled for the ARM architecture. This introduces new complexities however. The NDK still requires some Java as the compiled code is executed using JNI. In other words I still need to learn a minimal amount of Java and learn how to interface it with native code.

Finally, most of the information online assumes the use of Eclipse. Google provides some tools for creating ant build files but I have found the documentation to be lacking a bit. Through some trial and error I have figured out how to compile the NDK samples and in my next post will present a tutorial for doing so.

Topics so far:
* Creating an Android project from the Command Line
* Signing Android applications

Glow Artisan

January 8th, 2010 by Niki

A game that I worked on during my time at Powerhead Games was just released: Glow Artisan! It’s a puzzle game for the Nintendo DSi and it is quite addictive. It has already received a number of glowing (groan) reviews:

Nitendo Life 9/10
Game Set Watch

If you have a DSi I highly recommend forking over the $5 to buy it.

Updated dwm patches

September 9th, 2009 by Niki

After becoming increasingly fed up with Ubuntu I decided to reinstall Debian. I had used Debian for a number of years but eventually left it in favor of Ubuntu as I initially saw Ubuntu as Debian with more complete repositories. At first Ubuntu was quite promising however as I diverged from the default install I became annoyed at some of the idiosyncrasies of the system. After attempting to upgrade from Hardy to Intrepid (which requires a graphical installer – and all the directions/tutorials I found online assume you are using gnome/xfce/kde) I realized that what I really wanted was Debian with some additional external repositories.

In the process of setting up my system I decided to update dwm to the latest version (5.6.1). There have been some significant changes to dwm since 5.2 including better multi-head support. Even though I’m not currently using a multi-head system the changes affected my patches and they needed updating. The new diff files can be found on the dwm projects page.

New project: prime number storage

July 9th, 2009 by Niki

I have added a new project to the projects page: Prime number storage. When I was in college a friend of mine and I came up with a method for compressing and storing prime numbers. While doing research on our method we discovered that it was already in use.

However I recently decided to implement it anyway, as it was (and still is) interesting to me.