Put simply, topological information makes a lot of things possible for the intrepid ActionScripter.

E00 files non-redundantly store all nodes, lines, and polygons that make up a geographic data layer. This geodata format is one of three currently distributed by the Census Bureau for boundary files (the others are the shapefile and the Ungenerate ASCII format). The GIS formats used in most web mapping applications (I’m thinking of shapefiles, GeoJSON, and KML) are non-topological, meaning features are stored independently, and topological information on shared borders and the like is quite difficult to extract. Like seriously hard. Something you don’t want to be doing in the browser. Matthew Bloch, of the NY Times, did his cartography master’s thesis (at Wisconsin, natch) on MapShaper, much of which involved a C++ server-side solution for building topology from a polygonal shapefile. Generalization requires non-redundant polylines so as not to create gaps between features when smoothing. Other visualization techniques, including cartogram construction and graph decomposition, also require knowing the shared borders of geographic features.

Ideally, such topological information could be created/extracted for any geography, regardless of the datasource. In reality, topology building is intensive and best suited to server-side processing. Using E00 files and my E00Parser lets you experiment with the visualization and cartographic techniques only possible when such topological information is known, without the expensive processing necessary to build it.

The code

I’ve gotten a ton of use out of Edwin van Rijkom’s SHP library. My noncontiguous cartogram, isolining, and political choropleth experiments relied on the code to load coordinate data in shapefile form at run-time, as did the early experiments that led to indiemapper. I’m hoping I’ll get just as much use out of this parser, for when adjacency information is critical to the visualization technique.

There are two main classes, E00Parser and E00Tools. E00Parser is based on the Perl extension Geo::E00 by Alessandro Zummo and Bert Tijhuis, with much aid from the (world famous) Arc/Info Export Format Analysis. There’s no way I would have attempted to write the AS3 E00 parser without Zummo and Tijhuis’ code, as theirs appears to be the only stand-alone open source code available for reading the format. Their Perl regular expressions were copied with few modifications, though I did fix an issue in some that was keeping their code from accurately reading certain sections of double-precision coverages. I wrote E00Tools to collect a handful of methods for working with the resultant data.

I setup a Google Code project for this work, as topology will likely form the basis for a decent amount of my cartographic experimentation in the near future.

• to browse the code, just go here
• the latest zip distribution is available here
• three examples are included in com.indiemaps.mapping.data.parsers.e00.examples

Oh, BTW:

ESRI considers the export/import file format to be proprietary. As a consequence, the identified format can only constitute a “best guess” and must always be considered as tentative and subject to revision, as more is learned.

(from the Arc/Info Export Format Analysis)

How to use

After loading your ASCII E00 file into a string, use something like the following to parse it.

The returned data object includes all information contained in the file, and can have as many as nine sections. Of most use are the arc (non-redundant list of polylines), pal (list of all polygons and their associated lines), and ifo (attributes and labels) sections. The exact structure of the returned object is described on the wiki here.

There are three sweet examples to be found in com.indiemaps.mapping.data.parsers.e00.examples.

Tools

E00Tools contains some methods for working with the resultant data of E00Parser.parse(). Included are methods for:

• Drawing all features
• Drawing individual features
• Getting a list of polygon IDs for all features
• Getting the centroid of a feature
• Getting the shared border length of all features and their neighbors

Key above is the idea of the feature. Michigan is a feature. Features are not directly encoded in E00 files like they are in other formats. In a polygonal shapefile, for example, each feature is encoded as a multipolygon, constituted of one or more rings of points. In E00 files, only polygons are directly encoded; feature information (which polygons make up which features) can be ascertained from the INFO (ifo) section.

Experimentation

I created these AS3 classes for myself, because I wanted to experiment with topological geodata in visualization and cartographic applications. This typically boils down to knowing which features are neighbors and how much of a border they share. The E00Tools methods getAllFeatureNeighbors and getAllFeatureSharedBorderLengths gives you easy access to this information.

Daniel Dorling popularized the circular cartogram form among academic cartographers, outlining the symbology most notably in his 1991 PhD thesis and 1996’s Area Cartograms: Their Use and Creation (available here in PDF form along with many other gems of quantitative geography). Dr. Dorling made Pascal and C code available. I ported it to Python, and began experimenting, mostly in vain, on a method that worked with a shapefile as input, but without the expense of building topology. It produced at best a pale imitation. Dorling describes the gravity model used to produce the cartograms in his dissertation:

The algorithm which was developed to create the area cartograms worked by repeatedly applying a series of forces to the circles representing the places. Circles attract those they are topologically adjacent to; the strength of this attraction being greater the larger the distance is between them and the longer their common boundary.

The algorithm thus requires the shared border lengths of all features and their neighbors. Producing this info is easy with E00Tools, but it seems kind of backward to parse my geodata in ActionScript only to produce the rendering in Python. I’m working on porting Dorling’s algorithm to AS3 so I can go directly from geodata to cartogram without switching platforms.

Lee Byron mentions another technique, used to generate the Olympic Medal Count cartograms he helped produce for the Times. Byron didn’t release any code, but notes that a soft body force directed layout algorithm written in ActionScript was used. I haven’t been able to reproduce his method, but I’ve included an example that drops the topological information gathered from an E00 file into a Flare visualization using a force directed layout. The example is minimal, but shows how the E00 classes can be integrated with the Flare visualization API, and may point the way to a slightly different method for producing circular cartograms client-side.

As a first blush, the state level is alright (sorry Alaska, Hawaii). Here I’m showing the proportion of the dollars donated to the major candidates that went to Barack Obama.

Compare that blue and purple beauty, with only Mississippi to be embarrassed of, with this — the results of the popular vote.

Some of those states were obviously more consequential to the candidates’ finances. Here’s an interactive cartogram, sized by either votes or total dollars. Both cartograms use the 10th densest state as the “anchor unit” (in both cases, New Jersey), so comparisons between the two are meaningful. I talk more about that in my post on noncontiguous cartograms.

The state view is too coarse. The obvious choice is the county level, but such aggregated data is not available from the FEC, nor from the NY Times Campaign Finance API, where I retrieved all the finance data for this post. The data are available as individual records, or as summaries requestable by state or ZIP code.*

So I wrote some Python scripts to retrieve and process all 32,800 ZIP codes available from the Times API. There are more ZIP codes out there, but perhaps they had no donations in 2008. This had to be spread over a few days, because the Times limits requests to 5000 per day per API key.

Thanks to the shapefiles available from the Census here I was able to map the proportion of donations to Obama from those 32,800 ZIP codes. But too many ZIP codes lacked donations, leading to an unsightly choropleth characterized by radical change and data-less regions. Best to aggregate to larger units (but smaller than the states above). The NY Times made some nice interactive campaign finance maps, candidate-by-candidate, and aggregated to sub-state regions (ex. “Southern Wisconsin”, “Eastern Shore and northern Maryland”). I’ve settled on a finer unit, the three-digit ZIP Code Tabulation Areas (ZCTAs) of the Census Bureau (an aggregation of ZIP codes based on their first three digits). These first three digits correspond to the sectional center facility of the USPS that serves the area. Though that sounds rather arbitrary, the Census Bureau has aggregated to such units in some of their data since 2000. The following shows donations originating in the 877 three-digit ZIP code regions of the U.S.A., using the same color scheme as the maps above.

As above, compared to the outcome of the popular vote (but by county):

I’ll spare you the ZIP code regions noncontiguous cartogram. Cartograms rely on the recognizability of features on the distorted image, and 3-digit ZIP code regions lack familiarity save when they happen to line up with county boundaries. A better technique in such cases is described by Andy Woodruff of Axis Maps:

It’s a standard red-blue map indicating the winner of each county in the lower 48 states, where the transparency indicates the population of a county. The many counties with low population fade into the background, diminishing their visual prominence. This is meant to accomplish something similar to a cartogram, where sizes are distorted to show the actual distribution of votes.

Their election maps adapt the technique of encoding uncertainty information in transparency initially suggested by Alan MacEachren in 1992 and refined by Igor Drecki in 1999.

Andy tells me they grouped counties into 16 opacity classes using the natural breaks (Jenks optimization) method. I do the same here for my ZIP code regions. This method minimizes the sum of deviations from class means, thus producing an optimal classification. Sixteen classes ensures the appearance of a smooth gradient of transparencies. I used R and the add-on package classInt to create the classification. Here then: finance compared to votes, with both opacitized by consequentiality (total dollars donated in one case, total votes cast in the other).

And here the same over a white background (thus switching the visual variable representing consequentiality to saturation).

I’ve said very little about what these maps actually show. I’ll let the maps do the talking on that, though please do contact me if you’d like the data used in these maps for your own experiments.

One thing I’ve neglected to mention thus far: all of the above graphics were produced with ActionScript 3, using just a text editor and the latest free Flex SDK. I used Python to retrieve and process the campaign finance data, OpenOffice to paste the processed data into the DBF files of the shapefiles retrieved from the Census Bureau, and R to classify the data. It’s pretty sweet that such visualizations can be created using only free tools and data.

update: As I toiled on my ZIP code detour, it turns out GeoCommons Finder was accumulating the data I craved. As described there: “The monthly individual donor data was downloaded from FEC (Federal Election Commission), geocoded and then aggregated to county level for the lower 48 states.” The data provided there by county will still require some processing and doesn’t cover the full range of the data presented by ZIP code region above, but the common county aggregation makes further comparisons with voting data possible, and I’ll show some bivariate maps utilizing this new data in the near future.

1868

The honor typically goes to Émile Levasseur for the diagrammatic maps contained in his 1868 and 1875 economic geography textbooks.

H. Gray Funkhouser (1937) wrote of these “colored bar graphs”,

squares proportional to the extent of surfaces, population, budget, commerce, merchant marine of the countries of Europe, the squares being grouped about each other in such a manner as to correspond to their geographical position

Interestingly, Waldo Tobler (2004) points out that the example printed by Funkhouser (above) was sized by land area and thus not a true value-by-area cartogram. I don’t have access to Levasseur’s texts, and it’s odd that the only available scan of Levasseur’s first cartogram shows a diagrammatic map, not a true cartogram.

1897

On the other hand is the image below, whose units are definitely sized to the data, but whose geographic arrangement is questionable. I first saw this page from an 1897 Rand McNally Atlas of the World in a SpatialCollective post; a high res version is available from the David Rumsey Map Collection.

Circles on the left are sized proportional to population, those on the right to debt. Though the arrangement seems haphazard, geography is not ignored as the circles are grouped together by continent. I don’t really buy these as cartograms, but they’re certainly a predecessor to the circular cartogram form popularized by Danny Dorling nearly 100 years later.

1903

Others refer to the election maps of Hermann Haack and Hans Wiechel as the first cartograms (presumably because the popular example of Levasseur’s earlier technique was scaled by land area). These cartograms, which show election results in the Reichstag, were sized to population and of the same rectangular form as the earlier Levasseur diagrams.

Though these maps are mentioned in the second volume of Eckert’s touchstone history of cartography, Die Kartenwissenschaft (1925), they’re not reprinted there, and I’ve no access to the original sources.

1911

Professor John Krygier dug this one up — perhaps the first American cartogram, and an interesting example of the form.

This is the earliest non-rectangular cartogram I’ve seen of any provenance, and it is unique in maintaining the exact outer shape of the United States, while abandoning unit shape and position.

1929

Another early American example, the above map by Joseph Grundy was published by the Washington Post in 1929. States are scaled “on the basis of population and Federal taxes”. Though somewhat rough, I consider this the first modern cartogram, as it maintains topology without abstracting to rectangles.

1934

Erwin Raisz was the first to give cartograms academic attention, describing their production in “The rectangular statistical cartogram” (1934) and devoting significant coverage to the form in his popular cartography textbooks.

1961

A later example, but still a first. In 1961, Waldo Tobler arrived at the University of Michigan as an assistant professor and began working on the first computer programs for cartogram production. His “pseudo cartograms” were created by expanding or compressing the lat/long grid until the minimum root mean squared error of unit densities resulted.

Errors were typically quite high, though likely no higher than those on the manually-produced ones that came before. All subsequent cartogram algorithms can be considered variants of Tobler’s method.

Fully contiguous cartograms have stretched and distorted borders but perfectly maintained topologies. Like the Gastner-Newman diffusion-based cartograms we see all over the place. Though all sorts of cartogram designs have been produced, those with perfect topology preservation (fully contiguous cartograms) receive the majority of academic and popular press attention.

Some notable exceptions are the well done animated ones by Mapping Worlds and a recent NY Times example showing electors per voter that I’ll return to later. These fully noncontiguous cartograms preserve the shapes of enumeration units perfectly, but don’t even attempt to preserve any borders or adjacencies from the original map.

Judy Olson (Wisconsin Geography alum natch) wrote the only academic article to focus specifically on this cartogram symbology in 1976. She believed noncontiguous cartograms held three potential advantages over contiguous cartograms (I’ve three more below):

1. “the empty areas, or gaps, between observation units are meaningful representations of discrepancies of values, these discrepancies generally being a major reason for constructing a cartogram”
2. production of noncontiguous cartograms involves “only the discrete units for which information is available and only the lines which can be accurately relocated on the original map appear on the noncontiguous cartogram”
3. because of perfect shape preservation, “recognition of the units represented is relatively uncomplicated for the reader”

Despite these inherent advantages (along with ease of production), all the early value-by-area cartograms I’ve seen maintain contiguity. Some took the radical step of abstracting features to geometric primitives, like Levasseur’s early French examples (which may not have been cartograms) and Erwin Raisz’s early American “rectangular statistical cartograms”. But in many ways the noncontiguous design is the more radical cartogram, as it actually breaks the basemap apart — rather than skewing shared borders it abandons them.

my AS3 classes

Olson outlines a technique — the projector method — for manually producing such cartograms. A projector capable of precise numeric reduction/enlargement was required, but not much else, and accurate cartograms could be produced in minutes. A scaling factor was calculated for each enumeration unit, the projector was set to this value, and the projected borders were traced, keeping units centered on their original centers.

My AS3 NoncontiguousCartogram class works similarly. It takes an array of objects containing geometry and attribute properties and creates a noncontiguous cartogram. I include methods for creating the input array from a shapefile/dbf combo, but using KML, WKT, or geoJSON representations wouldn’t be too hard. Methods are included for projecting this lat/long linework (to Lambert’s Conformal Conic projection at least). The NoncontiguousCartogram class draws the input geography, figures the area of each feature, and scales figures according to their density in the chosen thematic variable.

It’s all good/in ActionScript 3, so can be used in Flash or Flex. The zip distribution includes the following:

• the main NoncontiguousCartogram.as class
• two example applications and the data needed to run them
• utility classes, including some that make creating cartograms from shp/dbf input quite easy
• Edwin Van Rijkom’s SHP and DBF libraries, which are used to load the shapefiles in both of the included examples
• Keith Peters’ MinimalComps AS3 component library, for the components used in one of the examples
• Grant Skinner’s gTween class, which is required by the NoncontiguousCartogram class for tween transitions

Browse all the above or download the zip.

Flash examples

I created a few examples, all started from just shapefile input. The following are screenshots, but there are interactive examples further down. First, an example out of Olson’s article, presumably produced using the “projector method” described above:

Here I’ve updated Olson’s example with more recent data and a snazzy red outline:

This one took a few secs to load the shapefile, project to Winkel Tripel, draw, and scale the cartogram by population:

The above is great as an artistic rendering, but is probably a bad depiction of the data as 10% of the counties have increased in size and there is a fair bit of overlap. Decreasing the alpha of the counties reveals the overlap, and creates a really interesting rendering of the data.

If you’ve read this far, you probably at least want to see some shit move around or something. Here’s an example I put together with election data; it shows off the basic updating and tweening capabilities built into the class. Notice how all features are drawn around their centroid and therefore remain centered as they change scale. Tweening is via Grant Skinner’s gTween and interface components are from Keith Peters’ MinimalComps library, which is great for prototyping in only AS3.

In the above, the “electors per voter” option is based on the same data as the well-made NY Times example I mentioned earlier, but something’s fishy about one of our scalings, because they don’t line up.

design considerations

Position and size. Both affect the one thing you’re trying to avoid on a noncontiguous cartogram: feature overlap. If features are left at their original centroids, overlap is likely if any units are enlarged. When sizing features on a noncontiguous cartogram, one feature is chosen as the “anchor unit”. This feature will stay the same size, and all others will be scaled up or down depending on whether their density is higher or lower than the anchor. If the unit with the highest density is chosen, overlap will be avoided because all other units will be reduced. This can often produce large areas of white space and units too small to recognize.

To address this, both the anchor unit and the feature centers are configurable. The “anchor percentile” is set in the class constructor, and can range from zero to one. If set to one, all but one feature will be reduced; if set to zero, all but one feature will be enlarged.

By default, features scale around their original centroids. You can pass a Dictionary of feature centers (using the objects contained in the combined array as keys) to the constructor (or via the settable property, featureCenters), which allows you to manually position the features. I include the following example to give you some ideas for how to use this feature. I create an interactive cartogram on which features can be moved around. Well Known Text representations of the feature centers are generated and converted into the required Dictionary via some included utility methods. Check out the source of this example to see what’s going on.

projections

In her 1976 article, Judy Olson used linework projected with an equal area projection. This was a requirement of her method, as she was using known land area measurements in her calculation of densities and scaling factors. My class calculates areas of features — projected or otherwise — before scaling, so any projection can be utilized and an accurately-scaled cartogram will result. In the examples above, I use a conformal projection (included in the source) on the points before passing to the cartogram class. Conformal projections preserve local angles (shape, sort of) and may result in features that are easier to recognize.

That said, for most applications it’s probably a good idea to use equivalent (equal area) projections before constructing these cartograms, as the divergence of the thematic variable from land area is often the point of creating the cartogram in the first place.

more advantages

In my thesis research last spring, noncontiguous cartograms performed quite well: subjects rated them highly on aesthetics and could locate and estimate the areas of features with relatively high accuracy. I would add the following to Olson’s list of noncontiguous cartogram advantages.

1. Olson concentrates on the perfect shape preservation of noncontiguous cartograms. The form (well, those with units centered on the original enumeration unit centroids, as in Olson’s projector method) also perfectly preserves the location of the features on the resultant transformed cartogram. Not only are features easier to recognize, but locations within the transformed units can be accurately located as well (cities or mountain ranges from the original geography can be accurately plotted on the transformed cartogram).
2. Because units are separate on the transformed cartogram, their figure-ground is increased and areas of features can therefore be more accurately estimated.
3. Many cartogram designs (including most manual cartograms and the Gastner-Newman-produced cartograms) sacrifice some accuracy for shape recognition. This is a defensible tradeoff, especially as area estimation is notoriously inaccurate and nonlinear. Yet it’s a tradeoff that noncontigous cartograms need not make, as they can always perfectly represent the data with relative areas without sacrificing shape preservation.

Thus, noncontiguous cartograms seem to excel at the cartogram’s two main map-reading tasks: shape recognition and area estimation. This is mediated somewhat by the chief advantage of contiguous cartograms: compactness. Because no space is created between enumeration units, contiguous cartogram enumeration units can be larger than those on noncontiguous cartograms, all other things equal. The increased size on contiguous cartograms may improves their legibility.

As with today’s pie charts, the area of each wedge is proportional to the figure it stands for, but it is the radius of each slice (the distance from the common centre to the outer edge) rather than the angle that is altered to achieve this.

I wanted to produce some just for kicks, so looked around for a script in AS3. No dice. OK, any language? Didn’t see anything. So I sat on the idea for a while and then finally thought up the technique that made producing them in AS3 quite easy. With the resultant classes, producing graphics like the following small multiples of U.S. soldier deaths in Iraq is a snap. The classes are written in AS3, so can be used with Flash, Flex, or mxmlc. All the example screenshots below are PNGs captured from SWFs produced with only AS3 (extended Sprites). To see the code (which includes a lot of ugly annotation), click ‘view source‘ below any image. All source code is included in the ZIP distribution linked below.

U.S. Soldier Deaths in Iraq, March 2003 to October 2008

view source

The above chart series utilizes the coxcomb in the same manner as Nightingale’s original — as a month-by-month temporal chart. The following is also a small multiples presentation, but utilizes categorical coxcombs to show car color popularity by percent manufactured.

Cars manufactured by color, 2003 to 2006

view source

And as the ultimate test for the classes, I decided to try my hand at reproducing Nightingale’s original graphic. I relied on Hugh Small’s beautiful reproduction of Nightingale’s graphic. I quickly gave up on reproducing the fonts, and indeed a few of Nightingale’s embellishments/idiosyncrasies are not reproduced in my graphic. But one of these nights I’ll extend the class to more faithfully reproduce Nightingale’s original. Nevertheless, not bad eh?

view source

I’m open to the criticism that these coxcombs (and other area-based symbologies) are rarely preferable to a simple bar chart (a length-based symbology). But I still believe they ought to be a part of the information designer’s toolkit, especially as they hold definite advantages over their more popular offspring, the pie chart. For animated or small multiples applications in particular, the coxcomb is preferable to the pie chart because of the constant position/angle of slices. It’s easy to glance across a series of small multiples coxcombs, or watch an animating coxcomb, and follow individual slices. I’ve built basic interactivity and tweening (using Grant Skinner’s lightweight GTween Engine) into the class. See this discussion over at Edward Tufte’s site for some of the advantages and disadvantages of coxcomb charts.

Further, though bar charts may be easier to read, they may not work in all contexts, including interactive mapping, in which compactness may outweigh other concerns.

Here’s an example of some of the basic tweening and interactive capabilities of the class.

view source

When I initially thought of coding these charts, I balked at the idea of constructing slices whose relative areas would faithfully represent the data. But I eventually realized that I could just use masked circles. The circles could be accurately scaled to the data, and since each segment would mask the same percentage of the circle, the resultant pie slices would faithfully represent the data. The technique (demonstrated in the diagram below) has the added advantage of allowing tweening of slices in animated displays.

the main technique behind CoxcombChart.as — the masking and rotation of CoxcombSlices

A bit easier said than done, since the strokes on each slice need to be drawn. But that boils down to geometry. Each slice need only be rotated into place to form the final rose chart.

I’ll create a few more examples in the future — I’m particularly interested in the cartographic applications of this multivariate chart. I’ll likely post a Modest Maps layer showing coxcomb charts of average precipitation-by-month for major cities, or something, in the near future. For now, here’s all the code used to produce the graphics above. It should be pretty easy to produce a coxcomb chart of any data using this code, and my interface is flexible enough to modify the appearance and interactivity of the charts for most applications. But feel free to extend the class and override methods to alter these properties.

• A ZIP of the full distribution. This also includes Grant Skinner’s lightweight GTween Engine (a single .as class) which is required for the class. Andy Woodruff’s DashedLine class is also included, as it is used in the Nightingale example.
• For you browsers, view all the source here

Creating the coxcombs should be pretty easy. Here’s an example instantiation of a non-interactive coxcomb, 350 pixels wide/tall, with 3 slices, each a different color.

Andrew Powell, Doug McCune, and Brandon Purcell have already posted great introductions to SpatialKey, so I won’t go through all that here. But just so’s you know: SpatialKey is a visualization system for geotemporal (location + time) data, developed primarily in Flex, that lets you filter and render thousands of points very quickly, all client-side in your browser.

This is not a formal release. We’re in a technology preview for now, which means you just get to see some sweet examples, but soon we’ll release a version, SpatialKey Personal, into which you can load and visualize your own data. Here are links to three of my favorite examples (for more, check out our Gallery page, or this post on the SpatialKey blog).

• where prostitutes are arrested in San Antonio
• the housing slump in Sacramento
• the radial spread of Wal-Mart

As I said, other better introductions have been written on SpatialKey; I just want to focus on a few of my favorite features or attributes.

not a single, do-it-all application

SpatialKey is based around a collection of visualization templates. Each offers a unique view of the data, with specialized visualizations, filters, and UI controls. Since the templates are specialized, each one is pretty easy to learn and begin using.

The examples linked above demonstrate the animation, map comparison, and drill down templates. The fourth template we’re showing off now is the temporal heat index template (here’s an example of that: Sacramento residential burglaries).

chorodot symbolization

You don’t see these much, but I think they’re really effective. The “heat grid” symbolization in SpatialKey is a modern implementation of a technique put forth by Alan MacEachren and David DiBiase in 1991.

Aggregating points to arbitrary but regularly-shaped polygons, or binning, was an extant graphical practice at the time, but the geographic application and their particular methods created an effective cartographic symbology. Other than SpatialKey, I haven’t seen this symbolization in a geographic visualization context, but I think it’s very effective at presenting large datasets that require aggregation. The heat grid symbolization in SpatialKey extends the approach by allowing grid renderings of attributes of the data (like house prices or temperature) in addition to aggregation of the count of points.

MacEachran and DiBiase’s example chorodot map of AIDS in Pennsylvania (image from J.B. Krygier’s lecture notes)

SpatialKey grid symbolization showing a data attribute (average home prices) in Sacramento county

small multiples / map comparison

I’ve always been a fan of the small multiples depiction of change, illustrated so well by Edward Tufte in The Visual Display of Quantitative Information and Envisioning Information. Though the SpatialKey Map Comparison template shows two multiples, it qualifies (and we can easily plug in more maps for specialized templates).

D.C. construction in the SpatialKey Map Comparison template

Both the maps and the time charts are live-linked. Mousing over an area on one of the maps or a bar on one of the time charts reveals the tooltip for both displays, allowing you to easily retrieve specifics for different time periods or areas.

complex temporal filtering and focusing via the heat index chart

The time chart, shown in the first screenshot above, is great for revealing linear temporal trends in a dataset, and for enabling linear filtering. But some datasets evince more complex temporal trends — for example, some crimes may be more common on a certain day of the week and at a certain time of day. Such trends are lost when data is aggregated in a linear fashion to, say, days or weeks.

sex crime arrests in Sacramento

The temporal heat index chart reveals such complex trends and allows filtering by multiple temporal aspects simultaneously (for example, showing only prostitution arrests on Tuesdays between 3 and 4 am).

in closing

I was late to the game on this one, joining Universal Mind in June. SpatialKey was developed by the brilliant team of Doug McCune, Ben Stucki, and Andrew Powell, led by Brandon Purcell and Tom Link, with product manager Mike Connor. It’s a privilege working with such a talented crew.

Our goals for this technology preview are modest (blowing minds, getting feedback), but we’re excited to continue developing SpatialKey and SpatialKey Law Enforcement. And we’ll be releasing updates, new examples, and SpatialKey Personal in the near future. So stay tuned to the SpatialKey blog, and please contact us if you have any feedback on our technology preview.

This cool app answers seemingly every question about when and how Manny hit his home runs…except for the one I’m most interested in: where on the plate is Manny most likely to hit a home run? I’ve seen a few visualizations recently that do this sort of thing, so I thought I’d share this view of current practices in the area of pitch location v. batting performance visualization.

ESPN will often show you a 1D view (showing only the x dimension of the pitch location) during games, dividing the plate into three horizontal segments (in this case showing the number of home runs this year by Vladimir Guerrero):

A more advanced, 2D view of the same is used often on ESPN’s Baseball Tonight:

For a while now, ESPN.com’s baseball Gamecasts have shown real-time pitch locations, including whether the ball was called a strike (red) or a ball (green).

Turning on “Hit Zones” reveals a very cool diverging blue-to-red heatchart-style graphic of batting performance in the 9 segments of the plate’s plane.

The above is apparently a common method of showing this statistic. These heatcharts differ in 1) how many segments the plate is divided into and 2) whether a sequential or diverging color scheme is used. Here’s one — with many more segments and a high swing zone — for Ted Williams, from the Official Ted Williams Website.

And a very cool report in the Baseball Analysts shows batting averages with 20 pitch location squares, aggregated over two seasons, and divided into the four types of batter-pitcher matchups. For example, the image below shows the greyscale heatchart for the matchup, Left-handed pitcher vs. Left-handed batter.

MLB.com’s Gameday does a better job, methinks, of showing pitch locations relative to the batter and the plate. But they don’t give any indication of the particular batter’s performance with different pitch locations:

I also like the above because it at least suggests the 3d nature of the strike zone: pitches with movement will not leave the space above the plate at the same position as they entered it. The heatcharts shown above do a good job of showing pitch locations in the x and z dimensions. A pitch visualization notable for showing these x and z locations at various points in y space is Lokesh Dhakar’s Baseball Pitches Illustrated (shown below is the slider), though this too is concerned only with pitching.

Thanks to the PITCHf/x system, there is a ton of pitch location data available. Currently, though, there have been few attempts to flexibly visualize this data. One app, the PITCHf/x tool by Josh Kalk, is quite flexible, but the charts themselves leave something to be desired (below for Ben Sheets).

I’m thinking more of a tool like Visual i|o’s baseball visualization tool, or the Boston Globe app linked above, that would take advantage of this rich data, but allow it to be manipulated and filtered in real-time. For example, I’d love to just look at called strikes or balls, and be able to filter that down by ballpark, or perhaps even by individual umpire, and visualize it by the ratio of strikes-to-balls, to get a better idea of the true strike zone. And, it’s worth noting that there’s no reason this information need be aggregated to grid squares; it could also be shown with a continuous density representation like this style of heatmap.

So, given a field of points (weather stations, say)…

…with one or more attributes attached (temperature, say)…

…a TIN can be constructed.

With the above TIN, values can be interpolated along each edge between the points of known values (control points). The interpolation is strictly linear (that is, the value 50 would be interpolated halfway along an edge whose control points were valued 48 and 52).

With a given contouring interval (I’m using 4 degrees F here), we can connect some of these interpolated points, creating our contour lines.

With the previous steps stripped away, this creates a passable isoline map.

The lines are rigid, though, and should be smoothed for presentation. I allow two methods for this. You can use the “simple” method, which just uses the built-in graphics method curveTo between the midpoint of each isoline segment (below with the isoline interval decreased to 3 degrees).

The above looks alright, but the curves are not continuous, closed loops can still have hard corners, and the isolines no longer pass through the interpolated points (we have therefore generalized an already-inaccurate interpolation). My compatriot Andy Woodruff, author of the glorious new Cartogrammar blog, offered to write a nice continuous curve method that ensured isolines would still pass through the interpolated values. You can read about the method in his post. Here she blows:

Bringing it all together, then, and incorporating the only extra feature I wrote (tinting of isolines), here’s a nice finished isoline map of temperature across the U.S.

My new isolining package for Flash/ActionScript3 accomplishes all of the above, requiring only an array of point data with attribute values attached. The above example, was accomplished with the following lines of code (after drawing the U.S. states from a shapefile).

The full example is included in the .zip distribution. Get that here:

• the full .zip thing (in addition to my isolines class and an example, the archive includes my delaunay triangulation package, Andy Woodruff’s cubicBezier class, and a slightly modded version of Edvin van Rijkom’s shp classes, which are used to draw the shapefile in the example .fla)
• or, for you browsers, the main IsoUtils.as class

Keep in mind: triangulation is just one interpolation method, and is many ways the least technical (and accurate). More accurate interpolation techniques include inverse-distance and kriging. ***If you’re having trouble, and your isoline interval is not an integer, check out the comment at line 171 of isoUtils.as. Please fix that, BTW.

I meant to add other features, but since I started work this past week, I’m posting the package as-is, and invite others to modify. On my wishlist:

• hypsometric tinting, or color between the lines, would allow for more effective terrain or temperature mapping
• support for projections and other coordinate conversions in the drawIsolines method. I have packages for converting lat/long to a number of map projections, but currently the drawIsolines method doesn’t have support for passing a point coordinate conversion method.
• an animated demo. This thing’s lightning-fast, so why not?
• something that would be super wicked would be if someone would implement Tanaka’s illuminated contours [pdf] method, that thickens/thins and darkens/lightens lines like so…
  
  …creating beautiful relief maps like the one below
  

If you add anything to the package, feel free to post a link to your revised version in the comments.

Now there’s nothing wrong with Blosxom (well…), nor with other minimal blogware, but I gradually grew weary of the quirks (for example, it took me a weekend to debug the commenting functionality) and I began to feel like it was taking away from my writing.

What does this mean for you?

• I’ve likely lost a lot of indie cred in your books.
• If you subscribe, you’ll want to update your feed to this.
• Outside links to individual blog posts won’t work. I can hopefully fix or redirect these eventually, but for now we’re all screwed.

On the bright side, you’re here, which means something’s working.