Blog

Learning from our users – first feedback on our prototype

In mid-April we publicly released the first version of our web application for feedback. We sought out advice from human rights researchers, international criminal litigators, investigative journalists, and policy advocates – spending over an hour with more than 45 users. Our whole team was struck by the willingness of our colleagues to dedicate time to giving us feedback. To everyone that took the time to talk with us – thank you!

This post will cover what we’ve learned from our users, how their feedback helps our mission, and why we’ll be reaching out again shortly.

Interviewees really liked… Interviewees had questions about …
  • The content itself
  • Charts showing command structures
  • Being able to access all the sources
  • Ways of going forward and backwards in time
  • Background information about security forces in a country
  • Showing analysis rather than just data
  • Visual clutter and data density
  • How to find data quickly
  • Downloading our information
  • Not knowing how data was selected for inclusion
  • Completeness of the data
  • Slow application responsiveness

We are answering the right questions, but not always in the right way

Users strongly validated the premise of our work – they very much wanted (and generally found it hard to find) information on the organizational structure, command personnel, location and areas of operation of security forces tracked through time. Interviewees could see the value of the dataset – in part or as a whole – to their own work. They were also intrigued by the visuals on offer, liking our ambition.

Our visualization of the command chart was a huge success, users loved being able to see that information in a visual way. Users had extremely positive views about actual data. They got glimpses its value, in particular where we included the analysis we can produce using the data. For example, for each alleged human rights incident we included a list of nearby units, which numerous interviewees found clarified the overall purpose of our research. They also liked the ability to see sources for each datapoint, which would help them appraise the data for inclusion in their own work. While users did not find the timeline functionality intuitive to use, when we explained how they could “time travel” through our data to see the command tree, commanders and other data at a particular point in time they were thrilled.

The biggest issue for users was that they were often overwhelmed with information, particularly in the initial map view of country. This made it difficult to grasp what they were looking at, or where to go next. There were many questions related to our terminology (what do we mean by area of operations, affiliated person, etc). Users often had difficulty navigating around the application from page to page, and our tools to sort through the visuals (like the filters on the map page) were not intuitive.

Users Want More of This Users Want Less of This
moreofthis lessofthis

What we will do next

The 45 people we interviewed told us straight what they liked and did not about our work so far They have given us good guidance on the direction we need to take.

  • Make it far, far simpler to use – less of a stand-alone application and more of a webpage
  • Simplify the presentation of key information, using visualization more sparingly, offering contextual guidance where needed.
  • Make it ‘search first’, giving lots of ways of find, sort and filter throughout the application.
  • Speed it up, and give lots of cues about how user things change things when the user does something.
  • Let people take the data home.

We have taken the feedback users have given us and will launching a radically new and improved application in the coming months – so stay tuned for more updates!

Why I started the Security Force Monitor

by Tony Wilson.

Today, I’m sharing something with you that I’m proud of.  It’s a new research tool we’ve created – a early version that shows our research, and embodies our reason for being.  It brings to light something we have never seen so clearly in one place before: the structure and operations of security forces, surfaced and arranged from thousands of publicly-available sources.

This slideshow requires JavaScript.

You can visit our prototype online here. Currently it covers security forces in Mexico and Nigeria – over 1,000 discrete organizations and nearly 900 affiliated persons.

How could all this information not already exist?

I started the Security Force Monitor in order to address a simple problem – the lack of detailed information about the police, military and other security forces of a country. At the time, I was trying to help advocates raise human rights concerns about U.S. security assistance to Bahrain. Since the protests that began in February 2011, rampant human rights abuses had been documented, but it was incredibly difficult for human rights researchers to identify specific perpetrators because the security forces were not transparent. This was extremely frustrating since it made advocacy supporting human rights conditionality on security assistance even harder. So, the task was clear: find detailed information about the security forces of Bahrain.

After several weeks of pilot research together with a colleague we had found data from hundreds of sources, and compiled them into an ever-lengthening Word document. This initial research demonstrated the information gap could be filled, but just as quickly a second problem arose – making sense of large amounts of quite detailed data. Using the limited skills I had, I created a rough Google Map and a rudimentary organizational chart, in an effort to make sense of all that data.

This slideshow requires JavaScript.

Even with these basic tools, I could see compelling connections between alleged human rights abuses and specific units and commanders. But the limitations were also evident. It was clear to me that in order to be a sustained effort the Security Force Monitor could not just work off of a text document or even a spreadsheet. It would need professionally-developed tools that a team could use to make accurate data easy to create, and a way of publishing it that would aid others in their own investigations.

Armed with some sketches of what a potential platform could look like, I interviewed almost 90 journalists, advocates, human rights researchers and others engaged in public interest efforts and asked them what would be useful to them and their work.

sketch
A capture from initial sketches for an application, showing a command tree, and a time slider

Their feedback guided the genesis of Security Force Monitor.  I have been fortunate to gain the support of the Open Society Foundations and the Oak Foundation, win the Knight News Challenge on Data, and pull together a great team – Tom Longley and Michel E. Manzur. The Security Force Monitor also found a welcoming institutional home at Columbia Law School Human Rights Institute and has begun to build an exceptional Advisory Council for the project. To move from concept to tool we have worked with the creative civic technologists at DataMade, FFunction and OpenNorth. Together we worked to create the datasets and first attempt at a product, the prototype that we are releasing today.

The picture it shows of security forces is rich and detailed, and changes with time. The prototype platform shows the changes that occurred over time as units were created, moved or disbanded; and as commanders were promoted, retired or fired. Finally, all our work is transparent: every data point is sourced with citations back to where we got our data.

This prototype is a big first step. When we were getting our idea off the ground we talked with almost 90 journalists, activists, researchers and policy experts. Now, with this prototype in hand, we will be will be conducting even more interviews over the next several weeks on what does and does not work in order to develop an even better version.

You may be getting an email from us very soon. In fact, if you have thoughts and feedback, email me directly – tony [at] securityforcemonitor.org.

Our mission is simple but bold. The Security Force Monitor will organize every piece of public information about security forces. We’ll produce research of the highest quality that will help make security forces more transparent. I believe that producing this research will aid journalists, civil society, human rights researchers, oversight efforts and others in making security forces more accountable.

You can help us do this. Take the first step by using our prototype and telling us what you think. Working together we can make the Security Force Monitor an indispensable digital service for transparency, accountability and human rights.