~traviscj/

Integrating Sparrow and Aperture/iPhoto

Fri, 13 Apr 2012 00:00:00 +0000

In my recent quest to take on digital photography, I've been using Apple's Aperture to store download my pictures and do basic manipulation on them. It's been working great, except that emailing is a pain because Apple hardcoded the email clients to:

Apple Mail.app
Eudora (really!?)
America Online(!?!)
Microsoft Entourage
Microsoft Outlook while I'd prefer to use Sparrow, a lightweight client that Sharvil got me hooked on a while back. Rather than continue to use Mail.app to send pictures via email, I found out that Aperture actually uses some AppleScripts to do the actual sending of the email. That got me thinking: I just need to change the script to mail using Sparrow, instead of Mail.

So all I needed to do is replace /Applications/Aperture.app/Content/PlugIns/Mail.applescript with the following script:

-- Mail from Sparrow.app instead of Mail.app

-- Travis Johnson (traviscj@traviscj.com)

on mail_images(email_subject, default_address, image_count, new_files, new_captions, new_comments, cancel_string)

	try

		tell application "Sparrow"

			activate

			set theMessage to make new outgoing message with properties {subject:email_subject, content:"Check out my sweet pictures!"}

			tell theMessage

				repeat with image_idx from 1 to image_count

					set this_imagefile to item image_idx of new_files

					set attachmentfilename to POSIX file this_imagefile

					make new mail attachment with properties {filename:attachmentfilename as alias}

				end repeat

				compose

			end tell

		end tell

	on error error_message number error_number

		log error_message & " " & error_number

		if the error_number is not -128 then

			tell application "Finder"

				beep

				display dialog error_message buttons {cancel_string} default button 1

			end tell

		end if

	end try

end mail_images

Now, magically, I can press the 'Email’ button in Aperture, and Sparrow pops up with a new email, and the proper file attached! Hooray!

I looked into doing the same thing with iPhoto; a slightly different proceedure is needed here. I opened /Applications/iPhoto.app/Contents/Resources/Scripts/Mail.scpt with Script Editor via the terminal:


$ open /Applications/iPhoto.app/Contents/Resources/Scripts/Mail.scpt

then pasted in the above code, clicked 'Compile’, then 'Save’. iPhoto then worked as expected, opening a new Sparrow message with the files attached.

chicago 2012 marathon

Mon, 02 Apr 2012 00:00:00 +0000

I've signed up for the Chicago 2012 Marathon with Team To End AIDS(full site or my donation page–PLEASE! every bit helps!). It's a bit of an unusual choice, probably, to not run for the American Lung Association(I have asthma) or the American Cancer Society(my dad is a survivor.) But it's actually been a good opportunity to learn a bit more about the current state of AIDS, AIDS policy, and what it's like to live with it. I've also had some really good discussions about it with people it would not have come up, so it has been nice all around.

P.S.: I will beat Oprah's time or die trying. (Post-dated entry so you know I'm not kidding.

fibonacci miles and kilometers

Thu, 29 Mar 2012 00:00:00 +0000

In my running, I have been trying to keep track in kilometers. This presents a couple problems: Somehow, my mind still thinks in miles, which is weird because I do not really have that good of an idea exactly how far a mile is, either. Or someone wants to know how far a 5k is. Or, when I was running the 2011 Chicago Marathon, there would be kilometer postings between the mile markers. Anyway, it is handy to convert between them, but a bit of a pain.

It turns out that the ratio of adjacent Fibonacci numbers are a decent approximation to the unit conversion between miles and kilometers. I decided to look into it a bit.

First, in case you have never seen or do not remember what Fibonacci numbers are, they are a sequence of numbers where the first two fibonacci numbers are and , and then each number after that is the sum of the immediately prior two. Mathematically, this is written as

which gives rise to the sequence

You can actually come up with a closed form solution for Fibonacci numbers, which is borderline incredible, using the recursion relationship given above. If you assume that , then you get

Then canceling a from both sides, and rearranging you get that

You can use the trusty quadratic formula to solve this, it gives

Clearly the Fibonacci number is not just this messy looking irrational fraction to the power, so we need one more trick. Introduce a couple of constants and , and let

Now, since we want and any number to the zeroth power is 1, we have that , or that . Then . The final step is requiring that . This gives that

Now, finally, we have that

It seems I have run a little off track. What does all of this have to do with running? Well, the Fibonacci numbers eventually approach this fancy we came up with earlier, and it turns out that . It also turns out that one mile is about 1.609 kilometers. That is just 0.56% error! Sweet!

There is a catch though. Some Fibonacci ratios are better than others:

	approximation	error
1/1	1	37.8%
2/1	2	24.2%
3/2	1.5	6.7%
5/3	1.666	3.5%
8/5	1.6	.58%
13/8	1.625	.97%
21/13	1.615	.37%

It looks to me like a sweet spot is at the 8 kilometer/5 mile mark. So now you can see that 25 miles is 40 kilometers, or whatever distance you wanted to know about in the first place.

digital photography

Thu, 29 Mar 2012 00:00:00 +0000

Recently I've decided to dedicate a slight bit of time to get a bit better at taking pictures. I'd always wanted to get a truly nice camera, and the opportunity presented itself, so I decided to go for it and bought a Nikon D5100 DSLR. I also ended up getting a complete steal on the Nikon 55-200mm DX VR lens, which was pretty awesome. Project 365 would be a great move from here, since I'm certainly no photographer, but it's been fun just learning about it all the ins and outs and how it all works together. In any case, I did start a Flickr: drtraviscj account which I'll be posting stuff.

More later

tjtestharness - a language-agnostic DVCS unit-test/continuous integration tool

Mon, 26 Mar 2012 00:00:00 +0000

I've been wanting to have a way to visualize which unit tests(or sets of them) passed for a given commit, if for no other reason than the sense of accomplishment from watching boxes turn yellow, then green, as they pass tests. The trouble is, I write code in a lot of different languages for a lot of different projects. I also don't want to bother with running unit tests individually–I want them to run as I perform commits.

My main project right now is working on a fast quadratic programming(QP) solver for series of related QPs, and in turn using that QP solver as the basis for a sequential quadratic programming method with a penalty globalization. It's working okay, but there's a lot of levels to the project, and it's easy to make a change in one place in the code that has far-reaching effects on other parts. It's also got a very complicated structure for testing, because of the nature of nonlinear programming. Testing runs something like

First, need to check the most basic version of the QP solver itself, and how robustly it solves the most basic QPs.
Next we start adding in bells and whistles: We use a couple different types of numerical algorithm approaches.
There are also two functionally identical but yet distinct approaches to actually formulating the QP.
A layer up from all of the QP solver, we're simultaneously building onto a nice penalty SQP method–which itself has many options and tweakable parameters.
A layer up from that, we're testing the penalty SQP method on many different nonlinear programs. They might require different settings in terms of tolerance parameters(standard tolerance might be difficult to reach in a given problem, for example.)
A layer up from that, we're also testing the penalty SQP method on mixed integer nonlinear programs. This involves solving many, many NLPs.

So there's a lot of levels and layers and levers and settings. What was happening is that I would change some parameter and eventually it'd get checked into the repository, and that wouldn't play nice with other codes.

I realized that I wanted some unit testing to automate this testing process, and document how it should be run, and soforth. But I also wanted the video game-style insta-rewards vibe to it, along the lines of the Panic Status Board(albeit less professional.) I also didn't want to spend a ton of time on it. I planned out the interface and databases over some beers with my roommate one night and wrote the entire thing in Django the next evening. I wanted it to run automatically on every repository commit. I also wanted it to try to make some sense out of Python, MATLAB, and C, since I use all three extensively. It should also actually be able to build a project from scratch, and every build should run from nothing–this enforces the rule that the repository is self as self-contained as possible. Finally, I also wanted to be able to look back at the actual run log of that unit test and see how or why it failed(and also have instructions on which repo/commit to checkout to see the bug in action).

My basic idea was that I wanted some program that kept a database of projects I'm working on, displayed some commits from those projects, showed the status of any test_*.{exe,mat,py} commands it could run. Okay, so I need a table of projects, one of commits, and another of tests. Originally I built it that way, but I wasn't sure how to get columns uniformly figured out. So I decided to split the 'test’ table into 'Tests’ and 'TestResults’, which worked out nicely.

The other tricky part was figuring out how I should add commits to the database, and how tests should get run. I ended up writing a fairly nice script for the commits to put in .githookspost-commit, though it requires the kludgy 'git push origin master’ in the post-commit as well. This is causing a bit of trouble with modifying commits(ie, it breaks the other script and causes extra merges/commits.) The not-so-interesting side was the one that peeks into the database, grabs the 'test’ (which is basically just a shell-script to run), and runs it. It also pushes the data back to the database when it's done.

Django is a really amazing framework. The entire coding part of the project went together in about 8 hours. I'm still not happy with it. I've got a laundry list of stuff to add here:

check for executable test_. (this is basically done, along with subdirectory test checking)
it'd be nice to move the actual unit testing to a server instead of my laptop. But maybe it's safer to not have the interface EVER web-pointing.
Big one: time/datestamp in the commit rows, to tell when you did it.
button to undisplay old commits, or a limit of how many commits show on the main page. This could get out of hand REALLY fast.
collapse commit sha1 (did this, should add: re-expand commit sha1's)
button to rerun tests(did this).

If you're interested, check it out at code.traviscj.com's tjtestharness

cplex matlab interface

Wed, 01 Feb 2012 00:00:00 +0000

Just for my own reference, I'm documenting the interface to CPLEX.

CPLEX expects a problem in the form

and is called by

cplex = Cplex('test');

cplex.Param.feasopt.tolerance.Cur = 1e-8;

if params.printLevel < 8

    cplex.DisplayFunc = [];

end

cplex.Model.sense = 'minimize';

cplex.Param.qpmethod.Cur = 1;

cplex.addCols(gk,[],bl-xk,bu-xk);

cplex.addRows(-ck, A0, -ck);

cplex.Model.Q = W;

cplex.Model.obj = g;

cplex.Model.lb = d_L;

cplex.Model.ub = d_U;

cplex.Model.lhs= c_L;

cplex.Model.rhs= c_U;

cplex.solve();

a trig problem solved in MATLAB

Wed, 01 Feb 2012 00:00:00 +0000

I came across this post. The basic idea is the guy wants to maximize constrained to this box, where is the length of beam . It's constrained to be a 61 cmx61 cm box, but one beam must start from 10cm up from the bottom right corner and the beams must meet at a point along the top of the box. I added the further assumption that the other beam must end in the bottom left corner.

which fall from simple trig. There's one more equation, which constrains the side length to 61 cm:

Next, I squared each pair of equations to get

and similarly .

I'm planning on using MATLAB's FMINCON, which means I need to formulate this as a minimization problem. This is accomplished by observing

Therefore, the final nonlinear program that I want to solve is

which can be solved with the matlab program

function xsol = solveproblem()

f = @(x) -x(1)-x(2);

x0 = [0;0;0;0]; LB=[0;0;0;0];

settings = optimset('TolFun',1e-8,'Algorithm','interior-point');

xsol = fmincon(f,x0,[],[],[],[],LB,[],@nonlincon,settings);

function [c,ceq]=nonlincon(x)

l1=x(1);l2=x(2);t1=x(3);t2=x(4);

c=[];

ceq = [l1^2-t1^2-51^2;

        l2^2-t2^2-61^2;

        t1 + t2 - 61];

end

end

When run, this produces a length 140.5 pair of beams. Hooray

spamfunc for optimization in matlab

Mon, 30 Jan 2012 00:00:00 +0000

what spamfunc is

In developing optimization algorithms, one of the most tedious parts is trying different examples, each of which might have its own starting points or upper or lower bounds or other information. The tedium really starts when your algorithm requires first or second order information, which might be tricky to calculate correctly. These bugs can be pernicious, because it might be difficult to differentiate between a bug in your algorithm and a bug in your objective or constraint evaluation. Handily, Northwestern Professor Robert Fourer wrote a language called AMPL, which takes a programming-language specification of objective and constraints and calculates derivatives as needed. The official amplfunc/spamfunc reference is contained in Hooking Your Solver to AMPL, but I'm shooting for a more low-key introduction.

You'll need to compile spamfunc before using it, but I'm saving that for last–until after I've shown off it's power a little bit. You'll also need a .nl file for spamfunc; these are generated by AMPL's command line program from an AMPL .mod source file. I'll be using rosenb.nl as my example here, for the Rosenbrock function.


>> [x0,bl,bu,lam,cl,cu]=spamfunc('rosenb.nl');

This serves as the initialization. It's worth noting the mathematical form AMPL bends your problem into. In particular, AMPL's spamfunc returns your problem in the form

So we've initialized 6 variables for our environment: the initial we specified in the mod file, lower() and upper bounds on those variables, the initial Lagrange multipler(lam, short for ), and the lower() and upper() bounds on the constraints. To get zeroth order information(that is, and ) for a particular point , just call spamfunc again with the the you'd like to evaluate at, as in


>> [f,c] = spamfunc(x,0);

To get first order information(the gradient of the objective and Jacobian of constraints), call spamfunc as


>> [g,A]=spamfunc(x,1);

so that the gradient is stored in g, and the row of A is the transposed-gradient . That is, the entry of the matrix is the derivative .

Finally, to get the Hessian of the Lagrangian, call spamfunc with just the multipliers , as in

>> WL = spamfunc(lam);

>> WF = spamfunc(0*lam); % if you just want \nabla^2 f(x).

Here, spamfunc reuses the last at which spamfunc(x,1) was evaluated at.

how to use spamfunc in your own code

Fine and good, but how do you use this in your code? I'll consider a very simple unconstrained optimization problem(and finally define the Rosenbrock function, if you've been waiting with bated breath). The Rosenbrock function–or, Rosenbrock's banana function, as I'll be calling it from now on, is defined as

If we wanted to use our spamfunc interface to solve this, we'd generate the .nl file as described below, then do a standard Newton method iteration. Since Newton steps are defined as and then the iterate is updated as , the most basic possible form of the algorithm is

[x,bl,bu,lam,cl,cu]=spamfunc('rosenb.nl');

grad = inf;

while norm(grad) > 1e-6

	    [grad,A] = spamfunc(x,1);

	    W = spamfunc(lam);

	    pk = -W\grad;

	    x = x + pk

end

Note: This is a terrible algorithm; there're no bells and whistles like line search, Hessian modification, etc. But it demonstrates calling at least a couple of spamfunc options.

how to compile spamfunc

I recovered an outline for building spamfunc and amplfunc from a project I've worked on; it should get you a compiled spamfunc binary. You need to make sure to have mex (MATLAB's compiler wrapper) in your path and properly configured.

wget --ftp-user anonymous --ftp-password traviscj@traviscj.com ftp://netlib.org/ampl.tar

tar xf ampl.tar

cd ampl/solvers

sh configurehere

mv makefile makefile.dist

sed "s/CFLAGS = -O/CFLAGS = -fPIC -O/g" makefile.dist > Makefile

make

cd examples

# following line should reflect where mex(1) is.

export PATH=/meas/bin:$PATH

mv makefile makefile.dist

sed "s/- o/-o/g" makefile.dist | sed "s/mex -I/mex -ldl -I/g" > Makefile

make spamfunc.mex amplfunc.mex

It's worth noting the difference between amplfunc and spamfunc; amplfunc returns dense matrices while spamfunc returns them in sparse format. MATLAB can do linear algebra on sparse format matrices much more efficiently than dense format matrices(as long as the matrices themselves are sparse enough!)

how to generate nl files

In the above examples, I started with an AMPL source file, rosen.mod.

rosenb.mod

var x {1..2};

minimize obj: 100*(x[2] - x[1]^2)^2 + (1-x[1])^2;

let x[1] := 1.2;

let x[2] := 1.2;

This file tells AMPL what how many variables you need, how you'd like to name them, what objective function you're minimizing, and what initial values they should take. To compile it to a .nl file, use


$ ampl -P -ogrosenb rosenb.mod

Congratulations! You now have a .nl file to feed into your super sweet solver!

VirtualBox and HRD/JT65

Tue, 10 Jan 2012 00:00:00 +0000

One of the semi-frustrating aspects of the state of computing and ham radio is the unavailability of some software packages on OSX. In particular, I've been interested in experimenting with JT65 and also wanted to use my Yaesu FT-857d's CAT cable to program memories and also try out HRD. (I should add–I've tried the JT65 stuff and some Yaesu memory programmers, but wasn't satisfied with the Linux setup for those. I'll probably look into getting them working more later, but for now, this article.)

In any case, I use VirtualBox with a Windows install to get support for these packages. The difficulty, though, is getting the Signalink interfaced with Windows programs. I've been unable to get them to interface directly, but one workaround is to:

This works, but there's another difficulty now: Your windows sounds will be broadcast over the air, which is a bit of a ’'lid’’ mistake. So probably turn off oyur Windows sound effects as well.

Congratulations! You have an OSX box running Windows running ham radio software!

tcjblog.py

Fri, 09 Sep 2011 00:00:00 +0000

I've now (for the most part) finished a working version of some software I've wanted to tackle for a while. I call it tcjblog.py. It's a blog platform built using jemdoc.py to format simple text files into nice looking HTML. What're the benefits of this, and features of tcjblog.py in general?

Simple, text file management of blog posts. (I control these with git.)
Ability to include LaTeX markup.
Static HTML for all blog-related pages
Entries show up on an index of all posts, the category associated with the post, and the month the post was made.
An RSS feed displays the formatted entries nicely for RSS readers.
Indexes split posts 10 per page.
Entry URLs are a sanitized version of the page title, so they're auto-perma-links, at least once you're happy with a title.

There's still a fair bit of work for it, but probably it's time to focus a bit on the content and less on the software, at least until it seems more pressing. This has been a great couple-afternoon project for me–a nice break from the almost 5000 lines of numerical/algorithms stuff in C I've done in the last couple of weeks.

It's worth noting that the static approach generates a fair number of files for this approach: one for each entry, one for each category, and one for each month, plus a tenth of the total number again for the index pages. So I use a makefile to manage all of it.