Software Vendor Evaluation Criteria: 50 Questions to Ask Before You Buy

Most software vendor evaluations fail at the same step: the questions. Teams ask about features and pricing, skip the hard stuff, and discover six months into implementation that the vendor has no documented data migration process, cannot produce a SOC 2 report, or charges 30% of contract value for professional services.

The questions below are organized by category. Not every question belongs in every RFP. Use this as a starting list and cut what does not apply.

Security and compliance (10 questions)

Security questions separate vendors who have thought carefully about data protection from those who cobbled together a compliance story for sales.

Do you hold SOC 2 Type II certification? Provide the most recent report. If no, describe your planned timeline.
Do you hold ISO 27001 certification? Provide your current certificate.
How often do you conduct third-party penetration testing? Can you share a redacted summary of your most recent report?
Where is our data stored? List all data centers and countries where customer data may reside.
What is your process for notifying customers of a data breach? What is your SLA for initial notification?
Who at your company can access our data, under what circumstances, and how is that access logged?
How is data encrypted at rest and in transit? What encryption standards do you use?
How quickly do critical vulnerabilities get patched?
Do you have a Bug Bounty program? If yes, provide the link.
Describe your disaster recovery process and your RTO and RPO.

How to weight these: Weight security at 3x minimum for regulated data (healthcare, finance, government). For internal tools, 2x is appropriate.

Implementation and support (12 questions)

Implementation is where most software deals go wrong. These questions surface whether the vendor has a real process or is improvising as they go.

Describe your implementation methodology. What are the phases and what does each produce?
Who will be our primary implementation contact, and what is their typical project load?
What is the typical implementation timeline for an organization of our size and complexity?
What does successful implementation require from our team? How many hours per week should we budget?
Describe your data migration process and your rollback procedure if migration fails.
What training do you provide for end users, administrators, and technical staff? Is training included in the contract?
What is your go-live support model? What happens in the first 30 days?
What are your support tiers and response time SLAs for each severity level?
Do you offer a dedicated customer success manager? What is the ratio of accounts to customer success staff?
How do you handle bugs after go-live? What is your priority classification and resolution timeline by severity?
Describe a recent implementation that went significantly over timeline or budget, what happened, and how you handled it.
Provide contact information for three references similar to us in size, industry, and use case.

What to watch for: Vendors who cannot describe their methodology in concrete terms probably do not have one. Asking for a reference from a failed project (question 21) separates honest vendors from defensive ones. See RFP Red Flags: Bad Vendor for more signals.

Pricing and contract (10 questions)

The total cost of software is always higher than the headline price. These questions surface where the extra costs hide.

Describe your complete pricing model: base platform fee, per-user pricing, usage-based charges, and other recurring fees.
What is included in the base contract, and what requires additional purchase (professional services, training, premium support, integrations)?
What are your professional services rates? Is there a minimum commitment in the contract?
How does pricing change as usage or user count grows? Are there tier thresholds that trigger an increase?
Describe your standard contract terms, minimum commitment, and auto-renewal conditions.
What are the fees for early termination?
How has your pricing changed over the last three years? What is your standard annual rate increase?
What happens to our data if we do not renew? What is the data export process and timeline?
Are there implementation or onboarding fees separate from the subscription? Provide a breakdown.
What integrations are included, and which require additional licensing or professional services?

What to watch for: Vendors who cannot answer question 27 often hide clauses requiring 90+ days notice to cancel. For a complete framework, see Total Cost of Ownership in Vendor Selection.

Product fit (10 questions)

Feature checklists are not product fit. These questions test whether the vendor understands what you actually need.

Describe how your platform handles [your specific core use case]. Walk through a typical workflow.
What are the top three limitations customers most commonly encounter?
How does your platform handle [your top integration requirement]? Describe the architecture, data flow, and limitations.
What customization is possible without professional services?
Describe your product roadmap for the next 12 months. What major capabilities are planned and when?
How do customers submit feature requests and how are they prioritized?
Are configuration changes after go-live self-service or do they require a support ticket?
How granularly can we control access by role, team, or data type?
What reporting does the platform provide natively? What gaps do customers typically fill elsewhere?
How does your platform perform at our scale? Provide benchmark data or references of similar size.

What to watch for: Question 34 is the most useful in this section. Vendors who name their own limitations are more trustworthy than those who describe a flawless product. Know the gaps before you sign.

Vendor stability (8 questions)

A great product from an unstable vendor is still a risk. These questions assess whether the company will exist and remain supported in three years.

How is the company funded? If VC-backed, when did your last round close and what was your runway?
How many employees do you have, and how has headcount changed over the last 24 months?
What is your customer retention rate? How many customers have you lost in the last 12 months?
Who are your key leaders, and how long have they been in their roles?
Is support handled in-house or by a third party?
Are there any pending acquisitions, ownership changes, or strategic pivots customers should know about?
What happens to our contract if your company is acquired?
Have you had any outages exceeding four hours in the last three years? If yes, what happened and what changed?

What to watch for: Question 43 matters most for early-stage vendors. A vendor with 18 months of runway and no path to profitability may not be around at renewal. That is not a disqualifier, but weight it accordingly.

Building a scoring rubric from this list

Once you have responses, score them consistently.

Categorize questions by weight. Security and compliance warrant 3x for most procurements. Product fit and implementation get 2x. Pricing gets 1x unless cost is your primary constraint.

Define what each score means. A 5 means the vendor answered fully with verifiable evidence. A 3 means they addressed it without specifics. A 1 means they did not answer.

Score independently. Have each evaluator score separately to prevent one person's enthusiasm from anchoring the group.

Flag dealbreakers before finalizing. A failed SOC 2 audit, an inability to provide references, or a 180-day cancellation notice requirement should eliminate a vendor regardless of overall score. Build a checklist and check it before finalizing your ranking.

For a deeper look at scoring, see How to Score Vendor RFP Responses.

Run the evaluation in Strutter

Building a scoring matrix for 50 questions across five vendors in a spreadsheet is a real project. Strutter lets you add these questions to an RFP, set weights per question, and auto-score vendor responses on submission. The comparison matrix shows all vendor answers side by side.

Try Strutter free at rfp.strutterai.com. Your first RFP includes full AI scoring and the comparison matrix at no cost.