Question Bank Audit: Keep, Rewrite or Retire Questions

A question bank audit starts with a familiar problem: your shared bank keeps growing, but nobody can explain why half the questions are still there. A once-useful prompt assumes an old workflow, two questions invite the same answer, and the reference answer rewards a phrase instead of useful reasoning.

TL;DR: Review each question against a current role and an explicit evidence goal. Keep useful questions, rewrite repairable ones, and retire questions whose purpose is gone; record the reason, owner and next review outside the question itself.

---

Start a question bank audit with one role

Choose one active role and level, then gather the questions your interviewers actually prepare for it. Include the question wording and its reference answer. Keep a separate list of questions whose use is uncertain so an unfamiliar question does not disappear just because the reviewer has never asked it.

Define the work this role needs to do before judging its questions. If that definition is missing, use a competency-based hiring framework to clarify it first. Here, a competency means a capability needed for the work, such as diagnosing service failures or prioritizing customer requests.

• Set a boundary. Start with the existing bank for one role, rather than redesigning the interview process.
• Name the decision owner. Ask the hiring manager to approve content decisions and a recruiting operations lead to maintain the record.
• Collect observations. Invite interviewers to bring examples of confusing wording, repeated clarification and answers the reference material cannot accommodate.

Tip: Judge the question and its answer together. Clear wording paired with an outdated reference answer still needs repair.

---

Use a keep, rewrite or retire worksheet

Copy this worksheet into a shared document or spreadsheet. It is an editorial record your team maintains manually, not a built-in Intervy audit screen. Give each question a stable reference so the decision remains understandable if its title changes.

Worksheet field	What to record
Question reference	Title or link, plus the wording being reviewed
Role and level	The current job this question serves
Competency	The relevant capability, stated in plain language
Evidence goal	What the answer should let an interviewer observe
Answer signals	Reasoning, actions and trade-offs worth listening for
Observation or concern	What prompted the review, with context
Duplicate coverage	Other questions seeking the same evidence
Decision and reason	Keep, rewrite or retire, with an explanation
Owner	One person responsible for the next action
Next review	A date and any earlier trigger
Completion note	What was changed, checked and communicated

Apply the decisions consistently. Keep means the question still serves the role, its wording is usable, and the reference answer describes acceptable reasoning. Rewrite means the evidence goal matters but the wording or answer needs repair. Retire means the question no longer has a justified purpose in this role's active selection.

• Keep with a reason. “Still tests incident prioritization under the current responsibilities” is useful; “interviewers like it” is incomplete.
• Rewrite with an action. Identify the missing constraint, ambiguous phrase or overly narrow answer signal.
• Retire with a destination. Record the replacement question or explain why the evidence is already covered elsewhere.

Key takeaway: Retirement is a content decision in this worksheet. Do not treat it as an instruction to delete a shared question before checking who still needs it.

---

Check relevance, wording and duplicate evidence

Read the prompt as someone outside your team would read it. Flag unexplained internal terms, missing constraints and requests such as “design the best system” that leave the interviewer to invent the problem during the conversation.

For stale assumptions, compare the scenario with the role's current responsibilities. An old technology name alone does not make a question obsolete: the underlying task may still matter. Conversely, a modern-sounding prompt can be irrelevant if it tests work this role will never own.

• Role relevance. Can the owner explain which current responsibility justifies this question?
• Answer quality. Does the reference answer allow defensible alternatives and explain the conditions that make them reasonable?
• Ambiguity. Would two interviewers need different extra facts before they could ask it?
• Duplicate evidence. If two answers were excellent, would they teach you anything different about the required capability?

Group questions by evidence goal, not just by topic. Two database questions might examine different capabilities; two differently worded customer scenarios might both elicit the same escalation checklist. Keep both only when the owner can name the distinct evidence each contributes.

Tip: Rewrite a duplicate's evidence goal before rewriting its wording. A new scenario with the same expected reasoning can leave the duplication intact.

---

Work through fictional before-and-after examples

The following examples are fictional editorial exercises for Lumen Labs. The questions, reference answers and decisions illustrate the worksheet; they are not preloaded product content or validated hiring measures.

Rewrite an ambiguous question and its reference answer

Before question: “How would you make our API faster?”

Before reference answer: “Add caching and indexes.”

For a mid-level backend role, the useful goal is diagnosing a latency problem before choosing a change. The original gives no workload, symptom or constraints, and its answer assumes a remedy without evidence.

After question: “A read endpoint becomes slower during the busiest hour. Error rates are unchanged. You can inspect request traces and database timings. What would you check first, and what findings would change your next step?”

After reference answer: “Compare slow and normal requests, separate database time from other work, and inspect the expensive operation. If repeated reads dominate, consider caching with explicit freshness requirements; if a query dominates, inspect its access pattern before proposing an index. Explain how you would check whether the change helped.”

• Decision. Rewrite; the diagnosis goal remains relevant.
• Answer signals. Looks for evidence before choosing a fix, states assumptions and explains how findings affect the next action.
• Owner and review. The backend hiring manager checks the revised wording before the next hiring round, then reviews interviewer observations after the pilot.

Key takeaway: A reference answer should describe useful reasoning, including alternatives. It should not turn one proposed solution into a required password.

Keep one question and retire a duplicate

Question to keep: “A customer reports an outage while another asks for help configuring a feature. What would you clarify, prioritize and communicate?”

Reference answer to keep: “Clarify impact and urgency, follow the relevant incident process, communicate the next step to both customers, and explain when to escalate.” For this fictional support role, the owner confirms that those responsibilities remain current.

Question to retire from this role's selection: “Two tickets arrive together: one service failure and one setup question. Which comes first?”

Old reference answer: “The service failure.” The reviewer finds that this narrower question repeats the first question's prioritization goal and adds no distinct evidence.

• Before the audit. Both prompts are selected, with different titles hiding the repeated goal.
• After the audit. Keep the richer prompt and its reference answer; retire the narrower pair from this role's active selection.
• Record the limit. Another role may still need the retired question. Check shared use before changing the shared bank.

Tip: Preserve the retirement reason in your worksheet so the next maintainer does not reintroduce the duplicate under another title.

---

Pilot the question bank audit before wider updates

Try revised wording in a small, planned review before replacing it throughout the bank. Begin with interviewer rehearsal using fictional answers, then agree which revision will be used in the next suitable hiring round. Avoid switching prompts silently halfway through a round where your team expects a consistent question set.

Ask interviewers to record observations about the question itself, separately from candidate evaluation. Use only the minimum information needed in the maintenance worksheet, following your organization's rules for interview information.

• Clarity observation. Which phrase needed explanation, and what clarification did the interviewer add?
• Evidence observation. Did the prompt invite the intended reasoning, or did it lead to a different discussion?
• Answer observation. Did a defensible response fall outside the reference answer?
• Timing observation. Could the interviewer reach the evidence goal within the time allocated to this question?

For example, a fictional pilot note might say: “Both rehearsal interviewers had to explain what ‘busy’ meant. We added the traffic context before using the revision.” That supports a wording change. It does not establish that the question predicts job performance or that the resulting hiring decisions are fair.

Key takeaway: A small pilot helps discover usability problems. Predictive validation requires a different kind of evidence; interviewer preference or a few smooth conversations cannot establish it.

---

Make approved manual updates in Intervy

In Intervy, open Questions in the relevant workspace. You can search the bank and narrow it by category and difficulty. Use those controls to find the question you have reviewed; the keep, rewrite and retire decisions remain in your separate worksheet.

Viewing questions and updating them require the relevant permissions. A person who can read the bank may not be allowed to edit it. If you cannot see Edit, ask the person responsible for question maintenance to make the approved change.

• Open Edit. Select the question's Edit action and check that it is the intended shared question.
• Update the pair. Revise Question Content and Example Answer together; check the title, category, tags and difficulty for consistency.
• Save and verify. Choose Save, reopen the question and confirm the saved wording before marking your worksheet action complete.
• Handle retirement deliberately. Record the decision outside Intervy and coordinate removal from future interview selections with their owners. Decide separately whether the shared record should be retained.

These are manual editing steps in a structured interview tool. The worksheet's audit outcome, next-review date and completion record are maintained by your team; this workflow does not depend on automatic audits, retirement statuses, reminders or bulk replacement.

Tip: Notify interviewers when an approved revision is ready. A saved question alone does not tell them why its evidence goal or reference answer changed.

---

Give the bank an owner and a review cadence

A technical interview platform still needs a content owner. Use a cadence your team can sustain: for example, review the active role's questions before each hiring round, discuss pilot observations after the round, and schedule a quarterly check of the wider bank. These are suggested editorial intervals, not product reminders or research-backed thresholds.

Bring a review forward when responsibilities change, a reference answer becomes disputed, or interviewers repeatedly supply missing context. Keep the original wording and the approved replacement in your editorial record if your team needs that comparison; do not assume saving a question creates a version history for this process.

• Hiring manager. Owns role relevance and approves the keep, rewrite or retire decision.
• Question owner. Maintains wording and answer signals, checks the saved change and records completion.
• Recruiting operations lead. Maintains review dates and follows up on unfinished actions through the team's usual calendar or task system.

Key takeaway: The useful output is a smaller set of justified, maintained questions with named owners—not a larger bank or a higher candidate score.

Start with one role and one worksheet. Use Intervy for the approved question edits, and keep maintenance decisions visible to the people who prepare interviews. For the surrounding preparation process, see the structured technical interview guide.