5 revealing questions for startup CEOs to assess their CTOs

Engineering is burning 50% of your start-up's budget. How sure are you that you’re getting value for money?

The CTO may tell you it's difficult to measure engineering performance. "Trust me, I know what I'm doing," says the CTO. And you feel you have no choice but to do just that.

These 5 non-technical questions are a litmus test.

They are not exhaustive, but they are designed either to be highly indicative of problems, or to justify, and even increase, your confidence in your CTO.

Note for CEOs: these questions are primarily for you to ask yourself, not the CTO. If you don't know, or cannot get the answer easily — that's a strong signal. These questions may naturally spark discussions with your CTO afterwards.

1. How aligned is the CTO with your business goals?

• Can the CTO articulate their technology strategy?

• Does it align with, and support, your business goals and long term vision?

• Is the CTO aware of your North Star business metrics?

• Can they tell you how engineering contributes to these?

2. Can your CTO bridge the gap between technical and non-technical teams?

• Can the CTO understand business concerns, and address them from a technical perspective?

• Can they communicate complex technical concepts to non-technical stakeholders?

3. What technology stack is your business built on, and why?

• The selection of a technology stack can have a significant impact on the business.

  • How was the decision made?

  • Were you consulted?

• What do you know about its architecture? Is it microservices? Why?

• Which cloud services does your platform depend on?

• What languages is it written in?

• Your company's innovation is probably not the tech stack itself, so ensure that it is comprised of widely adopted components.

• How easy is it to hire engineers with suitable expertise?

  • How easy will it be in 3-5 years?

---

4. What is engineering working on right now?

• Do you always know the answer to this question?

• Can you look it up by yourself and get a coherent answer?

• By the way, how does engineering know that these are the right things to work on?

5. How do you measure the engineering department’s performance?

"I have no clue how Engineering is performing." — many CEOs.

• There are lots of metrics, many of them useless or even counterproductive. Does your CTO know how to do it properly?

• Do your CTO report the statistics to you, like Sales and Marketing do?

• How long does it take to get a new feature into production?

• What percentage of deployments break things, requiring immediate remediation?

• What fraction of engineering resources are devoted to creating business value and moving the needle on your core metrics?

  • Do they routinely move the needle as much as expected?