Evaluting Tech Projects

Dirk's Guide to Evaluating a Tech Project available at:

Booki.cc/fundamentals-for-utilising-tech

Evaluation as part of the project -not something that you're doing at the end but something that's constantly a part of the project

How do you currently do evaluation? -what are the steps that you go through?

Brainstorm about steps to take for evaluation of a tech project now

What are the first steps to evaluate?

Some people are coming from the perspective of a community organization and some people are coming from the perspective of a consultant

• • • Dirk's Steps***

Based off a social justice frame.

1. Review/Determine Your Goals ---

You have to get people to articulate what they're trying to achieve and how the tech project will address the work that they're doing.

Suzi: Sometimes the goals aren't met by the building of the tool but you have to check in like 6 months out and see if the project was successful. So sometimes the success is outside of the project timeline.

Rebecca: I found it helpful to divide up goals into "what is the change you're trying to accomplish?" and "what are the goals of the technology to support those goals?" -mission-oriented goals and then technology-related goals that relate to those mission-oriented goals -then the technology providers can use the technology-related goals as the benchmarks for the project knowing that as those tech projects are finished they will connect to the larger mission-oriented goals

2. Determine its value ---

Getting into the tool itself and what it's trying to accomplish. What difference is it going to make? What is the benefit?

Jonah: that's great and it connects back to what Rebecca was talking about, connecting back to the work of the org.

e.g. if you're trying to make government data available to citizens, this tech project will create an app to make that data streamlined and accessible

Dirk: I do the Determine Its Value step before building.

Jonah: I do Steps 1 and 2 contracted because it takes so much time as an independent contractor and they don't pay for those steps. So much of the time, steps 1 and 2 come from the client (vs. having a deep set of conversations/groups about diving in)

Dana: we include time for this type of thing in the proposal. So first we have them commit to paying for the "discovery process" and then we have a proposal process.

Jonah: I like that model but i find it hard to believe that my poor-ass clients would be able to pay for that.

Dirk: Get the client to pay for a day or two to do this type of work before the project starts. They're paying you to put together the plan. Then, the client has a plan but they also have the option to go with someone else (so it's not like they're paying for nothing). Communicating is key to show them that this process takes time.

Jonah: From a business perspective, is that sustainable?

Dirk: People don't walk away from that.

Suzi: Yeah, because you're building that relationship with those people.

Many times small social justice organizations (for example) expect that consultants do Steps 1 and 2 for free or that consultants pop these proposals off really quickly and all the time.

3. Create Metrics ---

How are you going to measure success? We're often driven to work with metrics that don't exist in reality to pump up funding documents/proposals, etc. You do not want to do that when putting together an evaluative process for a tech project. Instead, you want metrics that will be easily be identified as to whether it was successful or not. These shouldn't be in funding proposal.

Also, you want to create timelines. When do you expect/want these things to be implemented/realized?

4. Create an evaluation plan ---

An articulation of "what is the process of evaluation?" that connects to the timelines of the metrics step. You want these to be very brief and light so that evaluation can easily be implemented into each step of the project plan. Make sure you've thought through if the results of the evaluations open up opportunities to streamline the project implementation. Make sure that it doesn't just happen at the end.

Using metrics and the timelines to check in on goals and initial motivation of the organization and project.

5. Implement ---

Involves lots of feedback loops

6. Start back over