This is really interesting because it changes the decision-making calculus for the person who is distributing the poster.   The traditional mechanism — take down unwanted posters — actually leaves the person who put the poster there net positive: it was up for some time and some people might have seen it.   But with the cancelled sticker, the person runs a risk that people will see it, be interested, and then believe it has been cancelled and not go.   This is even worse than not seeing it, because if that person sees subsequent posters, the “cancelled” message usually trumps and the person will still believe it has been cancelled.    This is a very creative way to solve the problem of unwanted posters.

But in the real world, it rarely is straightforward.    First of all, hypothesis tests rarely actually match the hypothesis.   The most common hypothesis test is known as the “nil hypothsis” — you are testing that a value is exactly equal to zero.  Yes, that’s right; most hypothesis tests are testing a continuous value against an exact point value.   Anyone familiar with probability densities should recognize that the probability of a value being exactly zero is vanishingly small.    Usually, what the hypothesis really wants to test is that some value is too small to be important.    BUT, researchers seem reluctant to specify what that “too small” value is.   The way we handle that is with the p=0.05 value; if there is less than a 5% chance that this value would be seen given that the true value is 0, then we call it “small enough.”

However, this misses another really important point: much research isn’t theory and hypothesis driven like that.  To see this, we need to elucidate the theory -> hypothesis -> test pattern a bit.   Usually, once you have a theory you have to find some data that can be used to test that theory.  Often researchers will conduct a lab experiment, and carefully measure the appropriate values.    Key word here is “appropriate”; researchers will figure out what kind of data they need, and then go get it.

Exploratory analysis turns this on its head.  You start with data, and then have to figure out what it is you really have.    You are severely constrained about how you operationalize variables.   You need to make arguments for why the operationalization you choose is a good one, and those arguments can’t be ‘convenience.’  You also end up trying lots of different statistics to see which ones make sense and which ones don’t.   This introduces a minor issue: if you have a 5% significance level, then you expect 1 in 20 statistics to be erroneously significant.  If you do 100 tests, you should have approximately 5 erroneously significant results.   But there are well-known techniques for dealing with this issue; I am more interesting in thinking about the other difficulties with EDA.

One challenge is understanding how to use traditional hypothesis tests, like t-tests.  With EDA, it is really easy to do LOTS of comparisons and look for statistically significant results.  But this misses the point; you aren’t just looking for correlations, you are looking for interesting relationships that suggest larger theories.   This means that you have to spend a lot of time looking at the results of each test, and figuring out what it is potentially saying.   What is this variable really measuring, and what higher-level concepts might it represent?   Look at the actual different in means, rather than just the p-value, and ask if what this difference represents; is it actually big and meaningful, or is it small or obvious?   Think about what higher-level concepts this might actually be testing, and then see if this really is a good test.  Often, EDA works by realizing that a given test you ran might be actually testing an interesting concept, but not really.  So you try a different test that actually does test that interesting concept — in this way, you move from one test to another to another, continually getting more and more interesting.

Another challenge relates to statistical models such as regression models.  In traditional theory-driven hypothesis testing, the statistical model is usually fairly straightforward; so much so that many statisticians assume that you already know what model you want and teach from there.  However, with EDA, you are doing more exploratory work, trying out different models and seeing how well they fit and explain your data, and what you can learn from them.  Complicated decisions, such as which variables to include, and what parametric assumptions to make (logistic, probit, poisson, zero-inflated, etc.) turn into multiple model runs and an attempt to interpret each variant.   Also, very importantly, it also requires work to figure out what good / interesting dependent variables should be, and a very difficult question is often “should this be an DV or an IV?”  In the end, when doing EDA you spend a lot of time trying out different models, including different combinations of predictors, and then spend a lot of time trying to interpret each model using hypothesis tests, graphs, and table to try to understand what the model is telling you.

Finally, another challenge involves understanding what variables you have.   Even this isn’t straightforward.  Often, you can benefit from post-processing some of your data to create new variables.   For example, if you have time-series data, you can create lagged variables that can help control for certain types of variation.   Often, it is beneficial to aggregate variables by subject and use that aggregation as a predictor.   Or to take a continuous predictor, decide on a cutoff, and make it a binary predictor (or multiple cutoffs and an ordinal predictor — few, medium, lots).

Exploratory data analysis is a distinct skill from traditional hypothesis-driven statistical analysis.  And it is used with different types of data.   I encounter EDA regularly when looking at existing log data from Internet websites; the data here is extremely rich and complex, but it requires a lot of work to figure out what you can learn from it.    Another place that seems to deal with this issue frequently is ecology; they collect lots of “field data” — and they often mean actual fields — and then use EDA techniques to study it.  I’ve found (mainly through the help of Emilee Rader) that ecological statistics can be really helpful in my work of understanding behavior on the Internet.

I find it fascinating how this works.  They basically made a minor modification to the “interface” of the shopping cart — they separated part of the cart and labelled it “fruits and vegetables.”  This caused people to buy many more fruits and vegetables than they otherwise would.   The online analog might be a separate box for “tags.”   Blog authors can always add keywords to the bottom of a blog post.   But by separately asking for tags, most people are more likely to contribute tags to their posts.

Now, Boice’s 4 step plan is actually quite interesting:

Step 1: Automaticity.  I wrote about automaticity last week.  The basic idea is to make writing, and especially starting writing, as automatic as possible.  Much like getting in a car and driving is very automatic, Boice wants us to make sitting down to write an automatic function.  He has some specific suggestions, which I discussed last week, such as spontaneous writing and generative writing.

Step 2: Externality. Writing is not purely an internal thing that you do solely because you love it.   At least, doing it for that reason alone doesn’t lead to being very productive.   Boice strongly suggests that the best way to be productive and make writing easy is to use external pressure to get yourself to write.   There are two types of suggestions he makes.   First, give yourself a positive reward for writing; make it so that the only way to get that reward is to write.   Don’t let yourself do something enjoyable that you do everyday (like shower) until you’ve completed a minimal amount of daily writing.   Another form of this advice is to track your writing; it feels good to put down a checkmark saying “I wrote today.”  Second, create an environment that is only for writing: a writing chair / writing desk in the corner of your office, or a room where you write.   And only use that environment for writing.  That way, you quickly and easily get in the mood to write when you enter that environment.   I find this last one difficult because I move around so much; I write from home, my wife’s apartment, my office at work, and occasionally the conference room at work.  However, I am going to try to configure at least my *software* environment in this way; software such as WriteRoom can greatly help with this.

Step 3: Self-control. In order to be productive as a writer — to produce a large quantity of writing — it helps to actually enjoy the process of writing.  If you don’t enjoy it and you are frequently dreading it, then eventually you will find a way to stop doing it regularly, even if you practice the other steps.   To combat this, Boice suggests that you work on controlling the thoughts you have about writing.  Don’t let yourself talk in your head about the difficulty or dreariness of writing; replace those thoughts with more upbeat thoughts about productivity and importance.   This is straight out of modern cognitive-behavioral therapy.  Replace the bad thoughts with good thoughts.  However, it is important that the good thoughts be true — be statements you actually believe.  So don’t try to convince yourself you really enjoy writing; remind yourself about the benefits of writing and the reason you are writing.   Surprisingly enough, if you take this seriously it actually can work.

Step 4: Sociality.  Share your writing, as early as possible, with other people.   Work on making writing a more social act.   Set aside a time to write with other people.  Share your early generative writing and first drafts with others for feedback.   Provide feedback on others.  In the end, it is hard to improve as a writer without feedback, and if you only get feedback on finished products then it is hard to improve the earlier intermediate outputs.   Making writing into a social act is what allows people to not only produce more, but actually improve the quality of the writing produced.

In practice, following all four steps can be rather difficult.  It takes a lot of willpower and introspection to regularly practice step 3.  In addition to the courage to share early work with others, it requires a lot of time and effort to find people willing to play the part of the “other people” in step 4.  Self-discipline and willpower are required in great and regular abundance to practice step 2.   In the end, following Boice’s 4 step plan is hard; it is at least as hard as writing in the first place.   But, it provides concrete advice; rather than “just write” it actually provides advice that is useful.   Also, it has been proven to work well for a wide variety of academics.  I still haven’t decided if this is the process I want to use to write, or what parts of it are worth adopting.  But it is certainly worth trying.

Automaticity to me is a really interesting concept.  The idea is that you make writing “automatic” — something that you become so used to doing that you can do it regardless of your mood.   Much like driving a car, you want sitting down and writing to be easy to do because you just do it.   To me, this automaticity reminds me a lot of the concept of “flow” from  Mihály Csíkszentmihályi.   Flow is a state of being where you are fully immersed in an activity and are doing it without distraction, and with both energy and control.  Most people have experienced states of flow before; it is often called being “in the zone” or “in the groove.”

Looking back, I’ve developed some strategies on my own for developing this kind of automaticity and flow.   The one I used most was talking to myself.  I’d start trying to explain something to myself.  Frequently, if I was excited by the concept I was explaining, I’d get really in to and the explanation would just flow forth from my brain.  Slight variants of this would be trying to explain something to my dogs, or to another person.   But I found that explaining it to myself often worked just as well.  Then, of course, the challenge was remembering it so I could write it down when I was done.  Often I’d lost parts of the explanation as I tried to remember it; I didn’t want to stop the flow in order to write anything down.

Boice suggests a different approach for achieving flow.  Really, he has two suggestions.   First, practice what he calls “spontaneous writing” and what some others call “free writing”: sit down and spend 5-10 minutes (use a timer) and just write.   Whatever comes into you head, write it down.   Yup, that’s right, write down that silly strange monologue that is running through your head.  Even if it is just “I don’t know what to write; my brain doesn’t work.” that’s OK.  Just don’t stop writing until the 5-10 minutes is up.  This helps achieve a bit of automaticity, but does it achieve flow?  It is hard to tell; usually I don’t really get into it enough to really be in a state of flow.

Next, Boice suggests “generative writing.”  In generative writing, you bring a little bit of the right brain back in, but only in certain ways.   You do spontaneous writing, but on a specific topic.  You let your right brain control the direction you are heading — steer your writing toward the correct topics — but you don’t let it edit or wordsmith or worry about audience.  Yet.   By removing the majority of the editorial functions, it frees you up in much the same way that spontaneous writing does.  But, I think this actually better produces a state of flow because I am writing about something I’m really interested in and want to see written.  And also because usually I’m writing about something that has been floating around in my head for a while.

Once done with generative writing, Boice suggests that you read back through it and see what you find interesting and useful.  Did you have a good way of explaining something?  Does the writing have a good structure?  You don’t have to keep it all, and you can certainly edit it later, but it is a good way of generating a first draft, especially when stuck.

Boice’s suggestions of spontaneous writing and generative writing are his attempts to make writing “automatic.” He believes that you can teach yourself to sit down and write automatically, then it is relatively easy to be very productive in writing.   One of the biggest difficulties that writers have is getting past that “blocking” feeling; not only does that feeling cause you to not write, but it also creates anxiety that causes you to put of future writing projects.   Writing in a fairly automatic way helps to eliminate that blocking feeling and get you moving; it also, therefore, helps remove the anxiety and makes it easier to start writing projects earlier (which is a good thing).

So, next time you are feeling blocked (or even if you aren’t), try out Boice’s suggestion for making writing more automatic: start by practicing spontaneous writing and then follow it up with generative writing.

A next-step theory focuses on individual user behavior in a given context, ignoring the fact that in a social media system, each user decision alters the context for future users.  Most lab and fIeld experiments in this area develop next-step theories because they control context, and focus on traditional outcome variables for individuals: did the user contribute? How much? Were the contributions high-quality? If we look at the evolution of a socio-technical system as a series of user decisions, a next-step theory only looks at how users make a single next step, not how the steps relate to each other.

While next-step theories are important, they are not sufficient if your goal is to design large-scale interactive systems.  Consider a next-step finding that users are more willing to contribute information to Wikipedia if they notice a current Wikipedia page is fairly short.  However, as users contribute, the context changes and the page grows in length.  Suddenly, the previous finding doesn’t apply anymore, and as a designer we can’t use that finding anymore.  Furthermore, users may anticipate that this will happen and decide not to contribute because they know a friend who also uses Wikipedia and would do a better job.  For these reasons, it is not clear how next-step theories *aggregate* to form system-level properties.  And is it usually system-level properties (sufficient quantity and quality of contributions, for example) that designers really care about.

The most naive way to move beyond the next step is to assume that the user context is approximately the same for future users, and therefore most future users will behave approximately similar to the next-step theories. I don’t mean to downplay this approach; many next-step findings do indeed hold across a wide variety of contexts, will hold for most future users, and can be used for design. But it’s not clear which next-step theories this will work for, and further testing is necessary to determine which theories are really useful for design.  Ling et al. (2005) illustrate the difficulty of determining this.

Looking to other fields, there are a number of techniques that can be used to move beyond the next step and study system-level properties.  Extrapolating next steps is difficult because there are numerous strategic concerns that can influence a user’s behavior.  Game theory is a field that formalizes the modeling and theorizing about how current user behavior both influences and is influenced by future user behavior.

Macroeconomics is another good candidate for potential ways to study large-scale interactive systems.   While most of the theories of macroeconomics probably don’t apply to social media systems, the research methods do.  Macroeconomics has developed many ways of measuring parameters of interest, theorizing and modeling their effects on the economy, and testing those models.  While the recent economic downturn shows that macroeconomics isn’t yet a resounding success, most of the discussion about the economy and possibly solutions were driven by macroeconomic research.  We can look to macroeconomics to better understand how to measure and theorize about large-scale systems that are too large and important to conduct rigorous controlled experiments on, but still need to be studied and understood.

Agent-based modeling, from the field of complex systems, offers a method to study emergent behavior in large-scale interactive systems.   We can model individual interactions and use agent-based models to study emergent properties such as critical mass.  Agent-based models are particularly useful when the interactions are too complicated to study analytically and the context too complicated to study empirically.

Finally, we should look toward more holistic inquiry into these large-scale systems.  Digital ethnography can provide valuable knowledge about the complex interactions inherent in large-scale interactive systems, and help us to understand the challenges and complexities in their design.

All of these methods require that we support and recognize the value of rigorous thinking: a valid and useful type of research involves rigorously thinking through potential outcomes (conducting thought experiments), and then theorizing and modeling possible outcomes.   This type of research can be time consuming, and doesn’t involve data or users, but is extremely valuable when studying large-scale topics.

Ling, K., Beenen, G., Ludford, P., Wang, X., et al. Using social psychology to motivate contributions to online communities. Journal of Computer-Mediated Communication 10(4) 2005

However, there are two aspects of risk that are think about: the probability of an accident and the severity of the accident.   Speed seems to be the primarily determinant of the severity of an accident — high speed means a worse accident.   However, speed is only weakly related to the probability of an accident.  Going faster only increases the probability of an accident slightly, and not always; when Michigan increased their speed limit from 65 mph to 70 mph, the number of accidents actually decreased.

I suspect that variance — how much faster or slower you are going relative to the average speed on the highway — is more important in determining the probability of an accident.   Think about traffic on the highway; if everyone was going the same speed (zero variance), then you only need one lane for traffic.   Driving is easy when everyone is going the same speed.   It is only the differences in speed that cause problems: slow people merging onto the highway, speed demons racing around traffic, and people slowing down to exit.   And it is these problem that often lead to accidents on the highway.  I suspect that variance — how much the speed of individual cars varies from the average — is correlated with the probability of an accident.   When there is a wider range of speeds, there is a higher likelihood of a traffic accident.  This could explain why the number of accidents went down in Michigan as they increased the speed limit; the people who drive the speed limit were closer in speed to those who drive fast.

Since this is a blog about incentive design, I like to ask what we can do about this now that  we know (or at least suspect) that variance is also really important.  First, I think that cruise control really helps.   There are really two main components of variance: how much my speed in my car varies over time, and how different my speed is from others.   Cruise control helps with the first one: my car stays approximately the same speed when I use cruise control.   That makes me more predictable, and reduces some of the variance that others have to deal with.   Also, since I try to use cruise control as much as possible, I’m usually willing to set it slightly lower than I want in order to stay behind someone else who is also using cruise control; this reduces some inter-car variance also.

Another suggestion: use one of those electronic signs that says “your current speed is X”.   However, instead of displaying one car’s current speed, display the average speed of cars over the last 10 (or 30 or 60) minutes.  One of the more robust findings in social psychology is that when people learn about how others are behaving, they move closer to the average.   Speeders tend to slow down toward the average, and slow people tend to speed up toward the average.   By posting the average speed, people will naturally reduce the variance in speed.  It is unlikely that this strategy will change the overall average speed, but reducing variance can have very positive effects also.

Given that driving places is both important to us drivers and important to the economy, it seems like a potentially bad idea to restrict traffic in work zones.   Yes, repairs and maintenance need to happen on highways; I really appreciate good highways and hate the damage that bad roads cause to my car.  But the roads are there for us to use, and losing time to work zones does real economic harm.  Right now, the western border of Michgan on I-94 has a 21-mile long work zone, complete with restricted speeds for that whole stretch.  If we multiple the time lost (~ 3 minutes if we drop from 70mph to 60 mph) times about 10 cars per minute, that means we lose in aggregate about 80 hours of person-time during the work day every day.  Do this calculation for all of the work zones in the US, and you realize truely how MUCH time drivers lose due to highway work zones.

Work zones are interesting because someone has to make 2 major decisions about them: how much area will the work zone encompass, and how much time will that area be a work zone.  These decisions are usually made in a way that makes it easiest (and cheapest and fastest) for the construction company to complete the highway work.   But these decisions rarely take into account the economic loss that work zones cause to drivers.  That job is usually left to the state transportation agency that pays for and oversees these work zones.  But they only have limited oversight; they usually cannot redesign work plans and are usually limited to accepting the lowest bid for a project.  As a result, many work zones are excessive.   To use that work zone on I-94 in Michigan as an example again, the 10 miles closest to the border is a nice newly paved 3-lane highway right now, and there isn’t a construction vehicle nor an orange barrel in sight, and yet it is still only 60 mph because it is a “work zone.”

How can we provide an incentive for construction companies to create minimal work zones?   We could enact rules that require very minimal work zones, but this wouldn’t necessarily be a good idea.  Sometimes it really is a LOT cheaper to tear up 10 miles at a time than it is to do it one mile at a time; equipment has to be rented and it is cheaper and easier to rent it all at once than repeatedly.  Redirecting traffic also costs a lot to put the redirection in place.  Sometimes it is more efficient for the state to allow a larger or longer work zone rather than pay for the extra costs of construction that a minimal work zone would require.

I’ve got another idea: fine construction companies for the economic damage from work zones.  Create a fine that represents the economic loss the work zone creates. If that 21 mile work zone causes 80 person hours of loss and the average Michigander makes $8 per hour, then fine the construction company $640 per day.

Here’s the trick: since the state is both the organization paying for the work and the organization collecting the fine, this tactic would be first-order revenue neutral for the construction company.   The construction company’s bid for the construction would include the cost of the fines in their bid, the state would give them the money necessary to pay the fines, and then the money would go right back to the state where it originally came from.   The construction companies aren’t really paying a fine, since they got the money from the state originally anyway. BUT, the fine would cause second-order effects; it would cause the construction company to alter their plans to reduce the economic loss to the state, because that would also reduce the fines they pay.  They can make more money (i.e. pay less fine) by reducing the size and duration of their work zones.  But, not if that dramatically increases the cost of the construction.  The company with the lowest bid will be the one that best trades off the cost of construction and the economic loss to the state (represented by the fines).

The biggest holdup with this plan is that the fines also shift some of the risk of construction from the state to the construction companies.   Construction projects don’t always go according to plan; weather can get in the way, or equipment availability, or accidents, or any number of other problems.  The fines increase the variability in the cost of construction; if weather causes you to spend an extra week on the project, then the fines increase your cost that much more.    However, if you are extra efficient and finish early, then you get to keep the extra money that you thought you were going to pay in fines as profit.  So this variability could be a good thing or a bad thing.  It isn’t clear how the construction companies would react to such a proposal.

However, being the researcher I am, I couldn’t help but think about the incentive design aspects of these work zones.  Given that you have a design goal of inducing drivers to slow down, what is the best way to do that?   The only design lever you have: signs on the highway.  So far, I’ve seen three strategies in use:

1. “45 where workers present.”  Allow people to drive fairly fast (usually 60 mph), but require cars to drive slower in areas where workers are present.  This allows people to still get where they are going relatively quickly without endangering highway workers.   Seems like it should work well, but only if people actually slow down.  In my experience, most people don’t notice the workers until its too late (generous interpretation) or just don’t slow down (cynical interpretation).
2. “Minimum fine: $600.” Threaten people with extremely high fines for violations in work areas.  Given that work zones are more difficult to police than regular traffic, and regular traffic has a low probability of being caught, these don’t seem to work that well either.
3. “Give em a break.” Appeal to people’s altruistic side.  Ask people to slow down for the sake of the workers.  If asked, I bet most people would agree to slow down.  But when faced with the possibility of being late, or having to spend even longer in the car with screaming kids, altruism turns out to be very low on the list of priorities.

So, what kind of alternative signs might better influence drivers to slow down?

People have a strong tendency to conform and emulate behaviors they see in other people like them.   This is called following a social norm.  Often, just informing people of how others are behaving can have a strong influence on behavior.   Robert Cialdini, the social psychologist, calls this form influence “invisible” because people rarely believe that social norms are the reason for their behavior.  But his experiments have shown that social norms have a very strong influence.  So one possible sign would be “The average speed of cars through work zones is 48 mph.”  However, Cialdini’s work has shown that information about other people like you has the strongest effect.   We don’t have much information about drivers to use to customize this message, but we do know one thing: which work zone they are in.  So an even better sign might be “The average speed of cars through this work zone is 48 mpg.”

One thing that seems to work well for police is those machines that display your current speed.  It makes people feel like they are being watched, and people who feel that way work harder to conform to expectations and rules.  On the highway, those machines don’t work as well because there are multiple lanes.  However, along the same lines, it might be valuable to put a sign up with “watching eyes.”  Often signs with eyes on them can induce that same feeling of being watched in people, which will cause them to slow down.   Again, this feeling and the associated behavioral changes might be somewhat invisible; even if people know what is going on and don’t think it will effect them, it does.

What are some other possibilities for signs to get people to slow down?  Leave thoughts in the comments.

Image credit: Nevada DOT

I have also successfully moved into a new house and purchased a new car with my wife for her to use.  She also got a job as an assistant professor at Northwestern University in the Communication Studies department.

Now that we are approaching the new school year, and I’ve got all this craziness behind me, I hope to be blogging more regularly about things I’m thinking about.  And I’m sure the new academic year will bring lots to think about: I’m teaching a class on the intersection of Computing and Journalism, and I’m running a seminar series on Social Media.  In the meantime, I’ve been traveling a lot, and spending that much time in my car has caused me to think about traffic more than I should.  So in the next couple of posts, I’m going to take a break from talking about social media and talk about how some of the ideas I work with can be applied to improve traffic.