Competitive UX Assessment

Our company faced declining market share for a profitable F&I product. Leadership needed to understand if user experience differences between our platform and a major competitor were driving this trend among automotive dealers. The study required a holistic view of the complex F&I ecosystem, including integration with multiple dealer management systems (DMS), lender relationships, and dealership workflows—all while maintaining objectivity and methodological rigor to provide actionable insights.

Strategic Importance

• Understanding UX factors in platform preference could inform product improvement priorities
• Insights would support strategic decisions about resource allocation and product development
• Identifying specific user pain points could enable targeted interventions to address market share challenges
• Data would inform ongoing competitive positioning efforts

Comprehensive Multi-Method Research Strategy

I designed and led a multi-faceted research approach to capture both breadth and depth of insights:

• Stakeholder interviews with Product Management, Operations, Competitive Intelligence, and Strategic Planning
• Performance Management panel to gather front-line observations and insights
• Deep dive sessions with dealerships representing diverse DMS environments and market segments
• Quantitative UX Longitudinal Assessment Program (UX-LAP) study with hundreds of respondents
• Analysis of product improvements and feature releases with corresponding results
• Evaluation of user behavior to understand the relationship between UX factors and business outcomes

Cross-Functional Collaboration

To ensure comprehensive perspectives and buy-in, I assembled and coordinated a diverse team including:

• Performance Management leaders to provide market context and dealer relationships
• Product Management to assess technical feasibility of potential improvements
• Business Services to understand lender-specific concerns
• Competitive Intelligence to provide market positioning insights
• Strategic Planning to align findings with broader business strategy
• UX Research to design and execute rigorous methodologies

Research Design Innovation

I implemented several methodological innovations to ensure actionable outcomes:

• Paired quantitative metrics with qualitative insights to identify not just what was happening, but why
• Designed a research sample that strategically covered critical DMS platforms and dealer segments
• Created a custom analytical framework to evaluate UX metrics by DMS provider, revealing key integration insights
• Implemented a factor analysis approach to distinguish between UX-driven decisions and other market forces

Jesse finds elegant solutions to big problems, whether the domain is product design, process design, or organization strategy. If you see an opportunity to add him to your team, pounce on it.

UX Comparison Insights

The quantitative assessment revealed:

• Both platforms met expectations for basic usability
• Neither platform fully satisfied users' product feature needs
• No evidence of a material UX disadvantage between the platforms in overall metrics
• Significant variations in UX metrics when analyzed by DMS integration context

Critical Integration Issues

The deepest pain points were data-related rather than interface-related:

• Data mapping issues between DMS and F&I platforms were causing significant frustration
• Specific field length limitations created usability problems
• Integration quality with dealer's DMS strongly influenced both dealer policy and individual preferences
• Different platforms showed strengths with different DMS providers

Choice Factors

The research uncovered key decision drivers:

• A significant percentage of users could choose between platforms
• Team member preferences significantly influenced dealer policy
• Most users who could choose had selected a single platform to use consistently

Data Integration Improvements

• Resolve known data gaps between our DMS and F&I platform
• Implement automatic identification of data mapping issues caused by "configuration drift"
• Address field length limits
• Enhance integration with third-party document systems

Implementation Strategy

Based on our findings, we recommended a phased implementation approach:

• Focus on data integration issues rather than UI redesign, as users were change-averse
• Prioritize fixes that would immediately address critical pain points
• Target users who have choice in platform selection
• Implement automatic detection of configuration problems to proactively resolve issues
• Conduct workshops with stakeholders to plan and prioritize implementation

Immediate Outcomes

• Shifted product improvement focus from UI enhancements to data integration quality
• Identified specific field-level issues for immediate resolution
• Established a clear understanding of the role of DMS integration in platform preference
• Created alignment across Operations, Product Management, and Engineering on priorities

Business Value

• Provided a data-driven foundation for investment decisions
• Identified a substantial opportunity segment of users with platform choice
• Enabled targeted intervention rather than broad, potentially disruptive changes
• Established metrics and baseline for measuring improvement impacts
• Created strategic clarity on competitive positioning

Methodological Innovation

• Established a rigorous UX assessment approach that has been adapted for other competitive analyses
• Demonstrated the value of combining quantitative metrics with qualitative insights
• Created reusable survey instruments for ongoing measurement
• Developed analysis frameworks that revealed previously hidden relationships between UX factors and business outcomes

• Integration Is Experience: In complex B2B ecosystems, integration quality often has a greater impact on user experience than UI design
• Contextual Analysis Reveals Truth: Breaking down metrics by relevant contexts (like DMS provider) uncovered insights that aggregate analysis would have missed
• Multiple Methods Matter: No single research method would have provided the complete picture needed for effective action
• Change Aversion is Real: Users strongly preferred fixing existing issues over redesigning interfaces, contradicting common product assumptions
• User Choice is Complex: Both personal and organizational factors influence platform decisions, requiring multi-level interventions