Ghost Hack

FITC Toronto Wrap-Up

Wed, 24 Apr 2013 20:37:00 -0700

Hi friends, I recently gave a talk at “Future Innovation Technology Creativity” in Toronto. The city was really cool, I loved the tech and design vibe, and the weed shop on Queen St.

I wish I had more pictures of the event. The whole time I was suppressing a back pain; must have pulled something at the airport. Which also unfortunately made this the most anti-social conference visits of mine, if I missed talking to you, apologies!

My talk was titled Creative Coding with THREE.js and I chronicle my year-long experience working with it, and why I think it’s an important piece of software and contribution to the web. You can >download my slides here<, though admittedly there’s plenty of omissions in the slides since most of my talk was spent looking at the interactive pieces and walking through how they were made and what they meant to myself and others.

For good measure, I’ll drop the link to THREE.js here. As always, credit goes to Mr. Doob and all the git contributors for making the library what itis to day: an excellent tool for creating visually awesome work on the web.

I really enjoyed Kyle Mcdonald’s talk on giving things away. When I saw the slide of his room, it reminded myself of my own barren mattress of a housing situation. He really is living the New York hacker lifestyle to the max.

I also enjoyed Matt Fisher’s talk on 3D-printing assault rifles. I guess when the playing field is even, maybe we can laser-cut some kevlar vests?

And finally, Daito Manabe is some sort of kefka-esque mad genius. Between the terrible (and often hilarious) translation from Japanese to English, and the insane things he does to himself in his projects… it was well worth the trip alone. Stay brilliant and weird, Mister Tokyo.

100,000 Stars Nominated for Webby Awards

Wed, 17 Apr 2013 13:59:08 -0700

100,000 Stars Nominated for Webby Awards:

Vote with me to help them win! - April 9th - 25th.

A few days left. Vote for 100,000 Stars, the project I helped create at Google Data Arts Team.

Also check out “Making of 100,000 Stars” I wrote for HTML5Rocks.

More to come soon :)

Sun, 09 Dec 2012 14:36:47 -0800

More to come soon :)

Timpulse: Who Works With 'Creative Coders?'

Tue, 16 Oct 2012 12:46:07 -0700

Timpulse: Who Works With 'Creative Coders?':

Colleague Tim Stutts posted an excellent article on “Creative Coders” and a list of agencies that often work with them. Also some nice discourse over at Hacker News.

timstutts:

Degrees and academic trajectories that combine design and programming as a way of creating digital media are becoming increasingly popular these days at educational institutions around the US and Europe. Students learn tools like Processing, OpenFrameworks, Cinder and ThreeJS to…

GAFFTA Workshop Introduction to WebGL with THREE.js Day 1 Part 2

Wed, 05 Sep 2012 21:02:00 -0700

In Day 1 we went over how to set up a THREE.js scene and how to create and add geometries to the scene. We also looked at the THREE.js api, and messed around with a Pong clone after watching this awesome talk called “Juice it or lose it” by some fellow indie game devs. Highly recommended watch, it’s entertaining, and pure awesomeness.

Then I shared this Pong clone I coded in THREE.js.

Download the pong clone code here.

View it online.

You were then asked to “juice up” this example as described in the above video, by adding particles, screen shake, color, whatever you want.

This example will show you how to actually do something with THREE, like a simple game. Let’s walk through the code real quick.

From main.js

playerOne = makePaddle();

makePaddle() from gameobjects.js is called, which looks like

function makePaddle(){
var geometry = new THREE.CubeGeometry( paddleThickness, paddleLength, 20 );
var material = new THREE.MeshBasicMaterial(
{
color: 0xffffff,
}
);
var mesh = new THREE.Mesh( geometry, material );
return mesh;
}

Here we see that I’m creating the player-controlled paddle out of cube geometries, then returning the mesh that gets created. This is an easy way to configure a mesh for creation and just add it to the scene for use.

You’ll also notice the material of type THREE.MeshBasicMaterial. There’s a bunch of material types you can play with. The important thing to note here is the passing in of a map with properties of the material. For example MeshBasicMaterial (see the reference code). Here we’re just passing in a color, which THREE expects as a hexadecimal value, like 0xffffff, rather than the CSS style hex value of #ffffff.

Let’s go back and look at main.js

playerOne = makePaddle();
playerOne.position = new THREE.Vector3( -500, 0, 0 );
gameWorld.add( playerOne );

playerOne is now a THREE.Mesh, which like I mentioned in the previous post, has position, rotation, scale. These properties come from THREE.Object3D which most THREE objects derive, like Light, Camera, etc.

Another thing to note is that position, rotation, and scale are all THREE.Vector3, which is THREE’s vector class. Here we replace the playerOne’s position with -500,0,0 which is left of center (with 0,0 being the center by default).

Finally, we add playerOne to the gameWorld which is another Object3D. Remember that THREE.js is a scene-graph, so we do this up above

scene = new THREE.Scene();
gameWorld = new THREE.Object3D();
scene.add( gameWorld );

So the scene graph looks like scene->gameWorld->playerOne.

The reason we created a “gameWorld” object is so we can hold all the game meshes, and if you wanted to, you could rotate or translate the entire game world. Try uncommenting this line.

// gameWorld.rotation.x = -1.2;

Play around with this project and try to modify it. Feel free to mail me questions.

Starting a Local Web Server

Tue, 04 Sep 2012 17:18:00 -0700

You may want to dynamically load things like OBJs, other code, or HTML to your projects. In this case you usually need to have a web server to do this.

While you could load your project onto say, ftp or dropbox, it’s much easier to dev locally.

First you need Python. If you are running OSX, you should already have it and don’t need to do anything extra. If you are on Windows, you need to download and install the Python Windows Installer.

Now, go to your command prompt or terminal (in OSX you can just use Spotlight to find Terminal, in Windows you use windows->run>cmd and hit enter), and browse to your project location.

Enter this command

python -m SimpleHTTPServer

If you point your browser to localhost:8000 now, you will see your project files or its index.html, hosted as if it’s on a web host.

GAFFTA Workshop Introduction to WebGL with THREE.js Day 1

Sat, 01 Sep 2012 15:37:00 -0700

Hi! This post will serve as your guide through the GAFFTA Introduction to WebGL with THREE.js workshop. This post is intended to be your companion for the workshop, in case you get lost on your experiments you can always refer back to this page for help.

In case you’ve landed on this page somehow and are not taking the workshop, you can also follow along.

THREE.js is a powerful open source javascript library started by Mr. Doob and contributed to by many amazing creative coders.

To get started with this workshop, you can follow along one of two ways:

Method A. This will get you good solid setup for any future experimentation you would want to do. I recommend this method first.

Download Google Chrome or Firefox, or any WebGL supported browser.

Download a good text editor so you can write code. For example, SublimeText.

Then download the example sketch. You can also view this sketch online. The example sketch contains all the files you need, such as a main.js where the code lives, and a copy of three.js in the same folder.

Once you have the sketch downloaded, you can open index.html in the sketch with your browser. That’s it! You should then see a rotating cube.

Method B. This method is totally experimental and it’s entirely possible to lose your work, but it’s really good for just trying something out. Give it a shot anyway!

Go to http://mrdoob.com/projects/code-editor/ Mr. Doob wrote a snazzy in-page code editor for the lazy. It’s still early in development and there are always issues with such things, so please keep that in mind.

You can go ahead and edit right in that window and update the code as needed.

Some handy links to have open when you’re working with three.js

The source repository so you can look at what each object expects as arguments.

The three.js wiki. This page might answer a lot of your questions.

Hello World

Let’s go over the formalities.

Three.js is a scene graph. If you are already familiar with OpenGL, it takes care of the render calls for you, you just need to parent objects to the scene and things will be made visible. If you are familiar with something like 3D Studio Max or Maya, then this hierarchy should also be familiar to you.

Let’s take a look at the main.js that comes from the example hello world zip.

Renderer

You need to instantiate a renderer. This will do the business logic of rendering your scene.

var renderer = new THREE.WebGLRenderer();
renderer.setSize( window.innerWidth, window.innerHeight );
document.body.appendChild( renderer.domElement );

Note that we’re setting the size of the renderer to the browser window’s width and height. Additionally, we’re attaching the dom element for the renderer to the body.

Scene

The scene is just a root THREE.js object. Things that you want rendered will need to be added to this object, like so

scene.add(myObject);

You will see an example of this further down the code.

Camera

Next, you need a camera. This will be where your render view will be looking at. For this example we do

var camera = new THREE.PerspectiveCamera( 60, (window.innerWidth / window.innerHeight), 1, 10000);
camera.position.z = 1000;

Here we’ve instantiated a camera with field of view of 60, an aspect ratio of window width divided by window height, and finally a near clip plane and a far clip plane. Don’t worry if you don’t understand this yet. These are what OpenGL expects when determining how to render your scene.

Now that we have the holy trifecta of THREE.js objects created, let’s put some geometry on the screen.

Geometry and Mesh

THREE.js has two constructs to be aware of, Geometry and Mesh.

Geometry is the thing that will hold on to vertex positions. For example, the vertices of a cube will be in geometry.vertices.

Mesh is a scene-graph object. Creating a Mesh usually involves giving its constructor a Geometry and a Material. We won’t worry about the material part yet.

So in our example

var geometry = new THREE.CubeGeometry( 500,500,500 );
var mesh = new THREE.Mesh( geometry );
scene.add( mesh );

We’ve created a CubeGeometry which is a cube primitive from this list of primitives. Then we’ve stuffed the geometry into a new Mesh. Finally, we add the mesh to the scene. The renderer will take care of the rest.

Animation

Finally, let’s take a look at our animation function.

function update(){
mesh.rotation.y += 0.01;
renderer.render( scene, camera );
requestAnimationFrame( update );
}

With mesh.rotation.y += 0.01, we’re telling the mesh’s rotation to increment every frame. Each Mesh is an Object3D that has position, rotation, and scale that you can modify.

Then following line, we make the renderer render by calling render() and passing in the scene and camera.

Finally, we do requestAnimationFrame, a nice and web-safe way of doing an animation loop. This causes the update function to be run over and over again so long as the user is viewing this page. Notice how we pass in the entire update function back into requestAnimationFrame.

Oh yeah, and update() needs to be called once, at the very bottom.

I’ll go over much more in the workshop, depending on the number of students and the level of knowledge attendees come with. Let’s make some webgl!

Generative Machines

Tue, 28 Aug 2012 14:42:00 -0700

A few months ago I was asked to make some visuals for Google IO , something I’ve done in the past with these procedural circuit-board diagrams. I spent a good few weeks experimenting with a ton of different things, in both THREE.JS and Processing, and slowly gravitated towards one concept.

This post will be image heavy, so I’ll skip on the failed experiments and get right to it.

Generative Machines

The idea of a procedural exploding view diagram was very attractive to me. It had all the right elements: a bunch of techy looking mechanical shapes, diagrammatic, and all of these lines criss crossing everywhere as if to suggest motion of a kind that automatically assembles.

It wasn’t enough that I simply had rivets and slots coming together, I also wanted to make an overall structure that looked as if, when assembled, became a machine that forms a whole that was more than the sum of its parts.

This got me thinking about creating procedural shapes that I can cut up. My first exploration into this was creating a one dimensional exploding axis, where elements can fly forward or back, and adding a radial dimension where elements could be chopped up and exploded outwards.

This involved in a lot of geometry splitting, helped by this wonderful THREE.JS library by Chandler Prall, based on a Constructive Solid Geometry (or CSG) javascript library by Evan Williams.

Example usage of this library.

Normally the library allows you to subtract geometry, add geometry, or intersections, however what I really needed was a cutting plane that slices any arbitrary geometry that I give it. I whipped up this hacky solution where I defined a plane, and a solver would create two gigantic boxes on either side. These two boxes are then used for two intersect operations on the geometry, one on each side of the plane.

This allowed me to have geometries with N number of slices, each forming individual mesh pieces that I can then animate.

While this got me somewhere, it still didn’t feel like it was creating a machine. It looked too abstract. What I needed was something that carefully straddles the line between hand-made geometry design and something that was procedural.

The next experiment involved procedurally generating shapes that were somewhat recognizable. Capsules lined with shells, screws, and wiring in this example.

Then I needed a skeletal base on which to generate the geometry. This would allow me to do replication (important in a machine-looking thing!) and some form of hierarchical generation.

I divided the code into three main layers.

The highest level was the skeleton, which would be a procedurally generated series of nodes (each using an Object3D, abusing THREE’s scene graph for my own purpose, more on this later).

The strategy I ended up with was to generate a “spine” from which radial and branching skeletons would emerge. After the skeleton was decided on, a function traverses each joint and generates the “meat” on the bones, called “assemblies”.

The next layer is what I call assemblies. Each assembly lives on a single joint length in the skeleton. They exist as functions that are hand tailored to procedurally generate something recognizable, such as a jet engine or a battery compartment (I wasn’t too focused on realism at this point). The assemblies get information about the joint length, and where the next joint is, but that’s about it. Inside each assembly are some randomized parameters, and sometimes conditions based on length, so each assembly will look slightly different based on initial conditions.

Finally at the ‘lowest’ level were components. These were essentially recyclable geometry generating tools I would use in each assembly to create the recognizable pieces, such as struts, lattices, wires, etc. For what it’s worth these were pretty quickly hacked up and not really meant to be reusable for the future, but they accomplished their job admirably.

Deciding on a color palette was interesting. I wanted the palette to be generated, but any ol’ random colors will not do. I needed sets of colors that work well with each other, but randomly distributed to the components.

I drew inspiration from these Mirror’s Edge 1080p wallpapers that I had laying around. The artists for this game really love color, and the palettes these images are blessed with are beautiful. So why not just borrow them?

I ended up creating these palette sets, based on sampling each image with approximate distributions. When the machine is first generated, a randomly chosen palette is picked. Each machine component type is then randomly assigned a color ID from the palette. This creates a look of ‘organized random’, since each assembly has these uniformity to them, even though the process for picking them was totally random.

Some more color palette examples…

Creating animation for these was one of the main features I wanted to try, essentially producing something that looked like it was assembling or dis-assembling based off of the exploding diagram they were inspired by.

I heavily abused these two wonderful tools, THREE.SceneUtils.traverseHierarchy, which crawls through the scene graph calling a function on each, and Javascript’s ability to give a function to any object.

Each Mesh that I want animated was given an update function, like so:

shell.update = function( percentage ){
	this.position.z = this.start + (this.end - this.start) * percentage;
};

The percentage would represented a normalized value between 0 and 1, 0 being a state of total disassembly, and 1 being total assembly.

Each mesh would then be responsible for taking care of their own translate or rotate values based on the input percentage, so I could easily set the initial and end state for the mesh. All meshes were procedurally generated with their end state, but their update function with a value of 0 would cause them to be at a disassembled state. For example at 0, a screw would be far off to the side, and by increasing it to 1 it would rotate and spin inwards towards its generated, final position.

I then used TWEEN.js to create interpolation to each animation. To create the staggered animating effect, I instructed the TraverseHeiarchy to go outwards from the lowest chain in the scene graph, animate those, and skip animating the parents (and their parents) until the children are completed with their animation. This causes the smallest, lowest bits of the assembly / component system to be animated first before animating larger entities.

There are some finishing touches that needed to be considered. I’ve experimented with putting the structure on black and dark gray, both of which gave an interesting look, but I still settled on white because of the black diagram lines, keeping the aesthetic more in line with the exploding view diagrams this was inspired by.

I added an ambient light color wash which took out a bit of the “CG Phong” look which plagued that shader when used by default, though admittedly it’s now become very instagr.am-looking.

I’ve also added some randomized specular lighting which gave it a metallic sheen. Finally, I experimented a bit more with the skeletal layout system to create some radial-symmetry, hopefully to create more orderly-looking constructs from a higher level, something like a Large Hadron Collider.

The finishing touches on UI and styling are by Geoerge Michael Brower.

Here are some more images from the final piece.

This project didn’t end up getting used for Google IO this year for reasons unknown to me, but I’m happy it’s seeing the light of day as a Chrome Experiment.

Arms Trade Visualization

Mon, 30 Jul 2012 16:05:00 -0700

*edit* This post was created a few days ago but had to be taken down. Should be okay now that It’s up on Google’s official blog.

—-

I work for Data Arts Team at Google and we get a lot of cool oddball projects here and there, more of which I might write about in the future, permissions pending.

Here’s something that is just wrapping up.

Arms Trade Visualization (best viewed on Chrome on full-screen).

A few weeks ago I was introduced to Dr. Robert Muggah, principle of SecDev Group who was giving a talk at INFO summit (Illicit Networks Forces in Opposition), a conference about how information technologies can expose shitty things happening in the world (slave trade, illegal arms dealing, bad actors network). Somehow our team was asked to make a data viz on data, and the project fell on my lap.

Of the data sets that were presented to me, one stood out the most ripe for visualization, and that was the (legal) arms trade data set collected by the UN and compiled by Robert Muggah and PRIO (Peace Research Institute Oslo).

This data set contains about a hundred thousand “trades” between countries for weapons reported by the UN. The columns represent the year of the trade (as high resolution as we could get, at least that’s what they said), the exporter and importer country names, the exporter and importer country codes (who knows what they are? I didn’t look very far), weapons code (UN specified) that refers to some categories like ammunition, military rifles, sporting rifles, etc, and the US dollar amount for the trade, unadjusted for inflation.

As an aside, I really enjoy data that comes in as “flat” as possible such as this CSV, especially for time series things. The trick is now to ‘bin’ this into something I can parse for a data viz.

The first order of business is to sanitize the country names against something I could use. Upon reflection I probably should have done this by matching the country’s ‘code’ id with some data base of countries, but my inquiry about this with the data provider didn’t turn out anything.

From my previous projects I already had a country list that had ISO-3166 codes matched to country names, for example “United States” to “US”, or “Taiwan” to “TW”. In addition, the country codes are paired up already with latitude and longitudes important for us later on.

Here are some examples of country names mismatching:

In the CSV: “Antigua & Barbuda”
In my set: “Antigua and Barbuda”

In the CSV: “Congo (Republic of)”
In my set: “Congo”

I decided it’s better to ‘fix’ the data set’s naming than it is to come up with yet another match list. Instead of a straight “find and replace” which would have been horribly messy especially with a hundred thousand entries and over two hundred and fifty countries.

I opened the CSV in Google Refine, a nice little utility that allows me to quickly sort and see issues in the data set. In addition to fixing the names, I could save out the “change-list” so that in the future if I found out I screwed up somewhere I could essentially revert or adjust, as the change list json acts as a modification history to the data (this comes super handy later when I discovered that I had missed a few countries).

The next step is to “bin” the data into groups by year, instead of attempting to parse a hundred thousand pieces of data each time. This is helpful since I knew my data viz was going to be a time series, and I knew that it would be useful to scrub the timeline and see changes, so organizing the data in a way that makes this easy was high priority.

I wrote a small python script to both convert and save the data out as JSON, which is really useful since I’m eventually going to parse it with Javascript anyway, and here I can organize it into the exact way I want to actually access the data. (Not sure if I can share this part since but it’s really simple).

Originally the weapon codes were described as such:

930100 – military weapons, and includes some light weapons and artillery as well as machine guns and assault rifles etc.

930190 – military firearms – eg assault rifles, machineguns (sub, light, heavy etc), combat shotguns, machine pistols etc

930200 – pistols and revolvers

930320 – Sporting shotguns (anything that isn’t rated as a military item).

930330 – Sporting rifles (basically anything that isn’t fully automatic).

930621 – shotgun shells

930630 – small caliber ammo (anything below 14.5mm which isn’t fired from a shotgun.

At some point we decided that the UN’s weapons trade categorization were far too arbitrary and confusing, even to the expert that we asked this about. So instead we just decided to combine the categories.

Okay! The JSON came out to be roughly the same filesize as the CSV, weighing at about 5 megs. Again, upon reflection, we could have used country codes to save space, but I wasn’t really worried at this prototype stage. I just wanted something on-screen so we can see.

You can see the output JSON here.

Time to parse this guy. I won’t cover too much on WebGL, THREE.js and globe code, but I’ll gloss over a bit of it here.

First I read in the country names, then matched up country name with country codes and latitude longitudes. Here is the json that I produced using public data found on the internet.

The following snippet is how I converted the lat lons into 3D geometry coordinates. The final code included a bit of fudging factor to make things match up, but for the most part this works.

	//	take the lat lon from the data and convert this to 3d globe space
        var lon = country.lon - 90;
        var lat = country.lat;
        
        var phi = Math.PI/2 - lat * Math.PI / 180;
        var theta = 2 * Math.PI - lon * Math.PI / 180;
		
	var center = new THREE.Vector3();                
        center.x = Math.sin(phi) * Math.cos(theta) * rad;
        center.y = Math.cos(phi) * rad;
        center.z = Math.sin(phi) * Math.sin(theta) * rad;  	
	
	//	save and catalogue       
	country.center = center;

Now we’re getting somewhere. Each line you see represents a trade. You can barely make out europe in that tight patch of countries where it’s really bright. The curvature is just a spline construction (see the code here).

Already it’s rendering slowly since *each* trade is a GL line draw call, so I decided to dump all of them into a vertex buffer, and it works much faster, but we have this laser ball now pointing to the middle. That’s okay though, since it will be covered up by an eventual sphere.

It’s also important to identify the trades by country, so here you see each country being assigned a random color and then applying the color to the line itself. By using additive blending and reducing the brightness, we can get a sense of volume for the trading area.

Next we need to see the countries! The most obvious way is to bring in some NASA texture map or a PNG of some country border, but that’s kind of boring, so I decided to use a “pin map” version of the world map presented by vertices. As you can see each country is composed of a series of vertices and I’m simply extruding them up to form hexagon bricks that I then color by the randomized country colors.

The data viz is now lacking a key layer of information… “which country is selling, and which country is buying”? How do I represent importers and exporters? I ended up with this experiment where particles represent weapon sales and are traveling from the country of origin to destination so you can see who are the primary exporters and importers.

Obviously you can only see how this works in motion, but you can tell from the final version that this solution is what I ended up with.

At this stage there’s still a few problems that I needed to solve. How do I represent categories of weapons sale, do I use color? Particle icons? Something else? Another problem you can see is the country centers. At this point they were using the country vertex locations and getting the average center, which is usually not what we think of as ‘center of a country’. For example, United States gets pulled north due to the amount of vertices Alaska contains. This was eventually fixed by doing a lookup of actual lat lons of countries instead of finding averages of vertices.

To prepare for the eventual UI that is going to be built, I prototyped the filter functionalities by adding DAT.gui, a simple UI interface made my friends on the team. This is really handy for getting quick checkboxes and dropdowns when doing creative coding work that needs to be adjusted on the fly.

A few things I decided to change in my design at this point, given feedback from the team and examining our options. First was that the pins (hexagonal bricks) representing the countries were too visually noisy, since the lines and particles were the actual “meat” of what we should be looking at. Second, the colors needed to be fixed so that they actually represent something useful other than simply showing different countries. Third, the particles themselves need a bit of love. Finally, I’ve also planned on adding some sort of 2D “HUD” markers that sit on top of the globe to act as selectors for the countries, in leu of doing actual country picking (more on this later).

The first problem was really tricky — I wanted to flatten the representation of countries back to a texture on the globe, but I also needed to highlight countries. Essentially I needed a dynamically updatable texture map, with any of the 250+ countries capable of lighting up when I needed it to.

A few considerations came to mind… I could render each country as vertex / quad geometry, but I would have to produce this geometry some how and that would be pretty time intensive and questionable if it would work out well.

Alternatively … I could have two hundred and fifty country image panels and somehow mapped their UVs for rendering. Again, this sounded really painful and I dreaded the notion of trying to map these correctly back onto the globe.

I tried this wonky solution of using an SVG of the world map (the equirectangular svg map found on Wikipedia), and using this cool little library called CanVG that renders SVGs onto canvas, and then injecting the resulting canvas into a GL texture.

It worked alright but the rendering time was around two seconds per render (!). It was unbearable, and given the level of interactivity I wanted this was unacceptable.

I talked to Ryan Alexander about this problem, asking if there was a GL shader solution. He recommended that I try a lookup table which I’ll attempt to explain here.

The strategy revolves around having a globe texture with each country colored at a different grayscale index. I wrote a small Processing app that took the SVG I was using previously and rendering each country (thank god the SVG was already split by ISO-3166 country codes!!) as a different gray value, between 1 and 256. If there were more than 255 countries I would run into problems… but thankfully that wasn’t the case. Additionally, pure black 0 was used to represent the ocean (no country).

In addition to outputting this image, I also had the Processing program write out a JSON array of grayscale lookup to country code.

Finally I cooked up this bit of GLSL code. The idea is really simple… there will be two image buffers, the first one holds the image above, and the second is a 256x1 pixel buffer that will be the actual color rendered for that country. For example, ‘FR’ (France) has a grayscale value of 3, so on the second image buffer I look up the third pixel from the left for what color to render France with.

With this I was able to mask any number of countries and color them instantaneously, all done from the video card. Using the same lookup map I was able to render one pass at the original grayscale (0-255 index) values to do picking, so that ended up being a two for one victory.

<script id=“globeFragmentShader” type=“x-shader/x-fragment”>
uniform sampler2D mapIndex;
uniform sampler2D lookup;
uniform sampler2D outline;
uniform float outlineLevel;
varying vec3 vNormal;
varying vec2 vUv;
void main() {
vec4 mapColor = texture2D( mapIndex, vUv );
float indexedColor = mapColor.x;
vec2 lookupUV = vec2( indexedColor, 0. );
vec4 lookupColor = texture2D( lookup, lookupUV );
float mask = lookupColor.x + (1.-outlineLevel) * indexedColor;
mask = clamp(mask,0.,1.);
float outlineColor = texture2D( outline, vUv ).x * outlineLevel;
float diffuse = mask + outlineColor;
gl_FragColor = vec4( vec3(diffuse), 1. );
}
</script>

The next problem I needed to tackle was the ‘country markers’, which were planned to include country names. Originally I used a crappy custom-written SVG-pinning library I wrote for shits and giggles. This allowed me to see what 2D HUD elements looked like on the globe.

The 2D elements are really dom elements since I didn’t want to render any type or solve those kinds of issues in WebGL, this is what the web is best for, after-all. The trick would be to translate the 3D position to 2D screen space position and then use CSS to place it correctly.

The following snippet does the conversion, using THREE.js’s projector class.

function screenXY(vec3){
	var projector = new THREE.Projector();
	var vector = projector.projectVector( vec3.clone(), camera );
	var result = new Object();
	result.x = Math.round( vector.x * (window.innerWidth/2) ) + window.innerWidth/2;
	result.y = Math.round( (0-vector.y) * (window.innerHeight/2) ) + window.innerHeight/2;
	return result;
}

then…

var matrix = rotating.matrixWorld;
var abspos = matrix.multiplyVector3( country.center.clone() );
var screenPos = screenXY(abspos);

and now we can give it to the element’s css style left and top for placement.

Eventually I replaced the crappy SVG lib with just straight dom spans as markers, and started adding typography so we can see the country names. There’s also a few tricks to figuring out the sizing, occlusion behind the globe, and LODing, but I won’t cover that here. You can see this in markers.js the (ugly) logic behind this.

If you’ll recall the color-space was being used initially to represent countries. Now that we have a much cleaner country separation, we could use color to represent other things. At first I tried using color to represent weapons categories, but this lead to problems due to additive blending, which tend to output a rainbow splash instead of anything visually useful.

We finally decided to distill (simplify) the color space down to two colors and represent purely imports and exports and upon reflection this was probably the wisest choice decision I’ve made in my career doing data viz. It’s still imperfect, since the colors do tend to merge (and I stubbornly refuse to give up on additive blending, see this image for comparison), but the end result strikes a decent balance between aesthetics and actual utility.

Our team hired Pitch Interactive to complete the remaining UI elements which ‘completes’ the visualization, including bar graphics for import export and category comparisons, filtering options, and a timeline scrubber with a historical graph showing change over time.

I wish I could talk even more on Robert Muggah’s investigation of the data, listening to him talk about it is really jaw-dropping since he can point out global conflicts directly on the visualization, for example in 1999 the arms race between India and Pakistan causing a gigantic spike in ammunition sales from Russia to India, or how depending on the country you can tell the exact brand of pistol or rifle (eg if it came from Italy vs Germany).

Here are images from the INFO event in Los Angeles featuring this visualization. Photos courtesy of Google.

Overall I’m pretty happy with the way the visualization turned out. It was a pretty classic “make this data look sexy” exercise, and as much technical hurdle as it was, the project itself was very very straightforward.

EyeO Wrap-up Part 2

Sat, 09 Jun 2012 21:11:00 -0700

When I ask attendees what their favorite talk was — most pointed to Robert (Flight404) Hodgin’s presentation. It was a real delight watching his new simulations presented and packaged up so neatly, lens-flare, lipsticks and all. I remember someone telling me off-hand that perhaps, of all speakers, Robert prepared the most of anyone. It really shows. I’m eagerly waiting for the EyeO video archives to be placed online so that you too can see his latest stuff.

One thing I want to note was his “Cornell box” that he presented all his simulations in. I often did similar things when writing software that required me to have a defined, three dimensional space — it was really helpful to get a sense of scale, to have a ground. Robert takes this to the extreme, creating a Portal (the Valve game) like “test chamber” environment, typography and all, with a fluorescent overhead light that could be turned off at will.

Another aspect of Robert’s presentations that I was really blown away by was the sound design he carefully paired the visuals with. Sound really does add 50% on top of the 100% rock solid visuals which were presented.

What really takes the cake was the humor and lighthearted-ness of it all. I don’t want to ruin it for you, but by the end of the talk I’ve really come to like party balloons.

Casey Reas’ talk was another highlight of EyeO. He had been the most important teacher in my university experience, and a guide to my subsequent career, so of course I had to sit front row and center. I sat there relaxed, and pretended to be once again in a UCLA D|MA lecture, soaking up wisdom and knowledge via osmosis.

Aaron Koblin also gave a talk slash interview. He showed some up and coming work from Google Data Arts Team that I’m part of, and was cool enough to give me some shoutouts and showed slides from some of my past work at Creative Lab. I’m still eagerly waiting for a lot of this work to be released publicly so I can write more about it.

Finally, I gave a quick five (ten? I forget) minute show and tell about my work with Google IO visuals and the sketches that led up to the final pieces. Hopefully in the next update or two I can talk more about them in detail.

EyeO Wrap-up Part 1

Fri, 08 Jun 2012 17:03:00 -0700

I’m privileged to have the opportunity to visit EyeO and hang out with the likes of Casey Reas and Robert Hodgin. It’s weird that our little “shiny” community has branched out and formed its own conference.

At the risk of rambling, I just want to quickly write that I haven’t blogged or tumblr’d or wordpressed or whatever for god knows how long. After having attended EyeO I felt that the best way to connect with these technocrats was to share the work that we all love, and also have a space through which I can write about the process on my own work, be it procedural art or game dev.

Marius Watz also gave a thought provoking talk, part of which I want to muse about. He was talking about this notion of over-used algorithms, eg vornoi, flocking, etc in our medium, and how it’s become a sort of cliche, a “found object” through which many artists simply snap screenshots and simply upload to flickr and call it a day. This reflected how myself and Aaron Koblin felt every time we’re tasked to make “yet another ” garbage flocking particle BS thing, and I feel like Marius summed up these thoughts quite well.

However… I also feel that 99.999% of the human population have no idea what a particle simulation is, or flocking, or iso-surfaces… as much work as it is that one puts into something highly complex, the layman, the non “technocrat” (as Marius puts it) will simply look at it and go “well.. that was cool”, and that would be the end of the dialogue.

I was talking to an editor Jason (…forgot last name…) from Wired Magazine during my lunch break. He laughed at this and compared it to comedians that either make jokes for other comedians, or jokes that are for the audience at large. I found this a somewhat apt analogy.

Another thought was that, our medium simply does not have an audience sophisticated, or been around long enough, to make comparisons to identify what works are simple snapshots of “found objects” and what works are deeper and more fulfilling. To make yet another comparison, film and cinematography has been around for a hundred years and there has been enough time for audiences that want deeper, more interesting stories, or films, or cinematographic techniques.

I asked my colleague George Michael Brower about this, and if I may paraphrase… “we are in danger of making ourselves more and more insular”, if we deny others to simply pick up and use “found objects”. He likens it to the chiptunes scene, where everyone in the audience also has their own chiptunes band.

to be continued…