We don't hate RFPs, not really

"RFPs might be a flawed process," said one of the participants at the STAN conference, where I recently gave a presentation about managing IT in a not-for-profit context. He went on, "but what you're proposing may be even worse."

What was this worse option I had put on the table?

I'm not sure exactly. That might sound odd coming from the guy doing the presentation, but bear with me.

The presentation was about the constraints that affect small organizations and not-for-profits when it comes to managing IT, and what the best approaches are for dealing with these constraints, in my opinion. It's a very big topic area, but I think I've done a pretty good job synthesizing things. You can read the summary handout for the talk and get an idea of the scope yourself.

RFPs, or Request for Proposals, are an approach to engaging IT service providers by providing some overview of what a project is expected to involve, and asking providers to give a quote. The RFP bullet point on the handout is just one little line on the lower right of the page: "Beware of RFPs – quotes aren't accurate, and too often they prescribe solutions instead of seeking to understand problems/needs." Every time I've done this talk, the RFP bullet has generated the most discussion. So I thought it deserves a blog post of its own.

And that's why I'm not exactly sure what my "worse option than RFPs" was; this is a topic that could be a presentation in its own right, and whatever short answers to questions I gave on the spot could not possibly address all the issues going through people's minds. I don't know what ideas people got from that part of the presentation, but here's my attempt to fill in the background a bit. I hope there's some lively comments to this post, because it is an issue that desperately needs discussion.

RFPs — only one part of the process

In the presentation I gave, one of my basic points is that there are a lot of things to do and think about, in order to use IT well and to have successful projects: creating flexible, expandable systems, thinking about the problems you're trying to solve before you prescribe technical solutions, standing up for what you know about your organization and its needs because technical partners will always lack your perspective on your organization, etc.

I know from experience that those are the things that are important for using IT successfully. But a lot of these ideas are new to the non-technical people listening to the talk. What is not new, is RFPs. In some cases, doing an RFP may be the only thing they do know when it comes to managing technology projects. "Put out an RFP and make sure you get three quotes!" They'll have heard it somewhere from someone, maybe a board member, maybe a fellow executive director.

So it's really not surprising that the one bullet that triggers the most engagement from the audience, is the one that they already know something about. They may have already put out RFPs, they may have even had some success with them.

Nonetheless, while RFPs may be familiar, they are just one part of the process. There are 12 other bullet points on that handout, and I think they're all of roughly equal importance. So one of the fundamental ideas I am putting on the table is that success in IT hinges on a lot of different things, not just the RFP.

Show me the money

It's also not surprising that RFPs are the thing people want to talk about, because let's face it, money talks, and RFPs are about money. They're a business tool for establishing relationships, and making deals. Nobody parts with their money easily, and the RFP process is essentially a formal way of trying to make sure the money only flows with the proper checks and balances in place.

But the bullet point says "quotes aren't accurate," and surprisingly enough, this is not an idea that people dispute. When I do this presentation, people often say, "we know quotes aren't accurate, but what else are we supposed to do?" I would say that it's fundamentally dishonest for two parties to knowingly participate in a process that involves fictional quotes. That process is bound to lead to bad decisions, and soured relationships. I am aware that's not an alternative. It's just a fact.

Why are quotes not accurate? Because it's the nature of IT projects that people rarely know exactly what they need at the beginning of the project. The page or two of bullet points in an RFP document, is no substitute for an ongoing series of conversations and feedback between client and consultant/developer. And through the conversation, everyone will learn about what the needs of the organization truly are, and what the technical possibilities truly are. Until that learning has taken place, no one can say for sure exactly what is going to get built in the project. If you don't know what's going to get built, it's impossible to put an exact figure on the costs.

But does that mean you can't quote on it? A typical tactic in IT consulting is to take your best guess at the costs, and then double it (and then double it again, depending on who you ask). Surely that much padding means that you can stand by a fixed price quote? Maybe so, but then I would say that the client organization is potentially overpaying, vastly, for the work that gets done.

Risky business

I think some people view the RFP process simply as a way of managing risk. There's so much more to it than just quotes, but nonetheless, organizations want fixed price quotes so they have a cost guarantee, and the risk of going overbudget is theoretically borne by the consultant/developer.

But is that true? What does a fixed price quote guarantee anyway? Chances are, if the consultant/developer incurs costs beyond what they quoted, they're going to come to you asking for more money. Certain large IT companies (that shall remain nameless for libel reasons) are famous for this. Government IT boondoggles are also a well known example.

When this happens to you, you could get really mad and point to the contract and the fixed price. The developer will no doubt argue that they have actually delivered what it says on the paper, and that what you're talking about is a new feature or a change request. If you end up arguing about what it says on the paper, then what you've successfully done is substituted IT and project management risk, with legal risk. And if you think IT is expensive, when was the last time you called a lawyer?

I think this whole idea of controlling the risk through the RFP is misguided. There is usually a difference between the RFP and the actual contract. Risks should be addressed in the specific terms of the contract, while the RFP can be more open and contemplative...you're seeking proposals, and not simply looking for the cheapest quote, right?

Contracts are, unfortunately, a whole other topic. For a terrific discussion about them, check out Peter Stevens' great blog post on different contract structures and what they suggest about risk and partnership.

Agile Project Management — part of the alternative

Agile project management techniques have emerged over the last 15 years, as a successful way of taming projects with high levels of uncertainty. At Freeform, we feel that the unique constraints that not-for-profits often face — uneven funding levels, high turnover and volunteer staff, lack of IT staff in-house, etc — all produce a lot of uncertainty, and so agile methods are a natural fit for the not-for-profit context.

Agile is a buzzword, to be sure, and there is a huge body of literature out there talking about what it means. But to boil it down, as we see it from a project management perspective, agile methods are techniques for ensuring that there is constant feedback between clients and consultants/developers, and also that there is a focus on delivering working, usable systems, as early as possible. It's a lot more valuable to a client if only part of a project is completed, but it actually works, than if only 80% of everything is complete, and nothing works.

Fundamentally, agile methods focus on ensuring that working features are delivered as early in the project as possible, and they keep being delivered, in short increments, throughout the project. All the work is divided into small iterations, often called sprints. After each sprint, the newly completed features are demonstrated, and there's an opportunity to re-plan and re-estimate future work, based on what everyone has learned so far. This way, changing priorities and new learning can be incorporated into the project as it happens. Instead of changes being disruptive to the project schedule, agile methods harness change as an expected, powerful force in the management of the project. The whole process repeats itself until the client is satisfied that the project is done.

Often, the "done" point is not where people originally thought it was. Since there's an emphasis on delivering working systems, early and often, many agile projects reach a point of diminishing returns, where the cost of further work is greater than the value of the few remaining things on the to-do list. Essentially, you can live without those last few things, so it's better to keep the money and spend it on something else. This is one way that agile methods put clients in the driver seat when it comes to determining the best use of resources on a project.

But we still need a website, and we've only got $XX,XXX to spend!

New and exciting ways of managing projects, to maximize value and incorporate learning, sound all well and good, but there is still that money thing. Budgets are not bottomless, and people still need to know if they can afford the work, at the end of the day.

The good news is that, within a certain margin of error, it is possible to provide real estimates for work, even when you only have the high level details of an RFP to go on. It helps if an organization is willing to talk to potential respondents to their RFP, to help fill in the blanks between the bullet points. A short conversation can go a long way to supporting good estimates. And in any well managed project, agile or otherwise, exact costs should be easier and easier to zero in on as the project moves forward. So you should have an opportunity to decide what you really want to focus on, and what can be jettisoned to cut costs.

This is why the new and exciting ways of managing projects actually matter, because they provide a way of incorporating the learning that happens naturally as the project unfolds, into the project management process, so you don't have to decide now, up front, which bullet points are in and which are out. You can discover that naturally over time, as you use a working system with gradually more and more features. You can make informed decisions about which high cost features really make a difference to your system, so you are sure to maximize the value for the dollars spent.

Measuring costs vs. setting prices

I think one reason there are lots of questions and concerns about this topic, is because of a lack of transparency in the business practices of IT companies. When the result of an RFP process is a bunch of numbers that somehow relate back to a bunch of bullet points, organizations can be forgiven for thinking that a website will cost $X, and a website with e-commerce will cost $Y. That is all that many providers are willing to say on the subject.

Essentially, some (many, most?) IT companies do quite well by hiding their true costs. Some organizations have spent a lot of time developing efficient processes for building the same kind of system over and over again, and as long as they can somehow contort a project to fit into that mold, they'll stamp their price on the RFP, and call it a day. By now, you can guess that I don't think that's a good process to follow, at least not for complex projects, with fuzzy requirements. If an "out-of-the-box" system could meet the needs of a project, then it doesn't really need an RFP in the first place, all you need to do is go price shopping and an RFP is a very inefficient and expensive way to do that.

Other organizations don't have cost structures that are controlled through repeatable processes. Instead, their costs are time-based. In this kind of situation, a fixed price quote is a way of making up for probable shortcomings in their time estimates. Remember the old trick of double, and then double again?

One way to think of time estimates is to remember what Michelangelo said of his statues: the form was already in the stone, he just had to remove the excess. IT projects may not be great works of art (at least not always), but there are parallels: a project is going to take a certain amount of time, we just have to remove the uncertainty, by doing work, and gradually the true amount of time will reveal itself.

So if the challenge is in measuring time for accurate estimates, is part of the solution in measuring time to produce accurate costs?

At Freeform, we have spent a lot of energy creating a sophisticated time tracking system that we use to log almost every minute of every day. The tracking system supports both invoicing to clients, and reporting on the work that was done. Each month, every client gets an itemized list that accounts for every moment we spent working for them.

This extensive measurement lets us be 100% open and transparent about what we're doing and what the real costs are for the work we do. Our time is our only "product" so when we're asking other people to pay for it, we consider it critical to demonstrate how it has been used. We only invoice for the actual work carried out. We cannot hide our work behind aggregate numbers on an invoice, and we feel it enables a more open and honest relationship with clients.

This is our answer to the criticism that what I'm proposing will lead to costs going out of control. Most organizations are not used to being involved in cost discussions with their providers, they're just used to getting a bill. But we believe strongly that if we're going to involve our clients in the project management process, and empower them to make decisions about the priorities, and incorporate learning into the projects, to achieve better outcomes that more closely meet the true, unanticipated needs that organizations have...part and parcel of doing all that is involving clients in managing the costs of the project. And we can only achieve that, if we have 100% transparency about the costs of the time spent doing each bit of work on the project.

It comes down to this: managing the time, and costs, on a project, is just as important as managing the implementation of features. They are in fact the same thing, when your only cost is the time spent implementing the features. This is a key aspect of the alternative I'm proposing: estimates can work when both parties are taking an active role in managing the work on the project, including the tasks and the costs.

What makes for a good RFP?

I'm not naive enough to think that RFPs are going away any time soon. They are a necessary part of due diligence for a lot of organizations. They do serve a useful purpose. I just think that too often the purpose they serve, and the way they serve it, is misunderstood.

So if it isn't clear by now, one of the main things that RFPs should not be relied on for, is fixed price quotes. This whole post is largely an attempt to explain why that is the case, and why that's not the end of the world. But what can they be relied on for?

RFPs are a way to start a conversation about your needs. Organizations should treat them as the start of the conversation, not the end. Nothing spells disaster like going through a rigorous RFP process, and then wiping the sweat from your brow, putting your feet up, and saying to yourself, "great, in six months we'll have that system we always wanted."

Instead, you need to remain involved in the project, the more involved, the better. So make sure you include questions in your RFP about the project management process an organization will use. How will they communicate with you throughout the project? How will they incorporate your feedback? How will they manage change requests? How will they make sure the work is on track? How will they report to you about costs?

An RFP is like a job interview. Do all the same kinds of things. Ask for examples of similar work/projects. Ask for references. Consider how long the organization has been doing this kind of work. How experienced are their staff?

Above all, talk to the respondents, even before the deadline for submissions, to help them put in the best response possible. Your bullet points might make perfect sense to you, but they won't make perfect sense to all the people reading them, so whatever you can do to fill in the blanks is worth while. You'll get more meaningful responses that will help you compare apples to apples when the time comes.

If you really want proposals (the P in RFP does stand for proposals, after all), then put the effort into describing your situation, not detailing a specific solution that you think you need. Don't say, "We need a website with a wiki for our member organizations to share information." Do say, "We have over 100 member organizations. They need to share information directly with each other about best practices for program delivery." Let the respondents propose a solution that they think will best meet your need; if you ask for a wiki, all you'll get is a wiki.

Give rough budget figures, or exact ones if you want. The more information respondents have about what your situation is, including the budget, the more realistic they can be about proposing solutions that you can afford.

The real bottom line

So what is the alternative to an RFP? There isn't a completely distinct alternative, but there are a series of ideas and actions that organizations can do, that will increase the chances of successful RFP processes:

• Remember that there's more to managing IT than doing a good RFP.
• Don't ignore the risks inherent in the project; the RFP process by itself won't make the risks go away (but the right kind of contract can help).
• Consider agile development and project management methods, to maximize the value for the money spent, and harness change as a productive, not disruptive force.
• Seek a level of transparency from your technical partner that you are both comfortable with; you need a productive relationship.
• Treat the RFP process like a job interview, not a product comparison.

It sure sounds like a lot more work than just getting three quotes and calling it a day, but I don't think it's any worse.

I tend to get worked up about this topic, because I find it gets in the way of what I really co-founded Freeform for. I do this work because I personally want to help organizations that do good work, and help them get the most out of the expensive investments they have made in IT. I want to work with people who are trying to make a difference, and help them do that better.

If only it were that simple! Still, despite the business challenges, the project management challenges, and the technical challenges, it all seems worth it when we build successful relationships. Successful technology is great, but I value the relationships more, because they endure even after the technology has outlived its usefulness.

Above all, an RFP process is a chance to start a relationship. It is the beginning of a conversation, not the end. I hope these ideas have made it clear that there's a lot more to consider than just the quoted cost, and maybe your next RFP will be the beginning of a beautiful friendship.