Together with members of the Agile Experience Design LinkedIn group, I created a survey on the State of Agile UX. The overarching goal of this survey was to begin to understand what is and what is not working when it comes to adoption of Agile among UX designers. The survey was inspired by Version One’s annual State of Agile survey, which have been surveying Agile adoption since 2004(?). Similarly, we wanted to establish a benchmark for what is likely to be an adoption that will continue (and evolve) over the years.
The survey was conducted between Oct. 16-24, 2010. We received 150 responses. We are still churning through the findings but I wanted to post some initial findings I thought were of interest. I am still working on a more detailed analysis of the data. If you have experience wrangling Excel (e.g. doing PivotTables) and doing survey analysis, pls do get in touch.
- Total Participants: 150.
- 80 participants identified as being primarily UX Designers.
- Survey Goals:
- To understand the level of success in integrating Agile and UX disciplines, and the reasons for the success or failure.
- To establish a benchmark for future surveys.
- 18% described themselves as very successful
- 45% as somewhat successful.
- Only 5% described themselves as very unsuccessful.
- The written responses at the end provide some insights as to the reasons behind the success or lack thereof.
- We received a lot of suggestions for how future surveys could be improved, such as to focus less on web design, focus less on user interface design, or on UX as a role.
Findings in a Nutshell: Same as it Ever Was
The challenges with integrating Agile and UX are not too different from the obstacles one might encounter when integrating a traditional UX practice into a waterfall methodology. Any form of change will meet resistance, logic and reasoning be damned. I remember years ago having to fight tooth and nail to convince management of the value of doing user research and creating detailed wireframes and specifications. The digital world has changed significantly since then, with detailed specs being far less-cost-effective. But resistance to change is the same as it ever was. And integration of Agile and UX is no exception.
Among the reasons for failure, it was therefore not surprising to see cultural resistance at every level (organization, team, peers) as being the strongest pattern. Another major pattern appeared to be going through the motions of change (e.g. doing something for two weeks and calling it a sprint or stand up during meetings and calling it a Standup), but missing the critical underlying Agile thinking, also seemed a strong pattern among those who described themselves as not succeeding.
Not being co-located, i.e. being in the same physical space as other team members seemed to also be a major factor. At the same time, some replied that they were successful despite being physically separated, because they were in regular contact virtually. Another interesting stated reason for failure was that Agile developers seemed to not regard the UX design role or discipline as valuable or important. Remember, it is not just UX designers who are part of this change; the attitudes of developers and the team as a whole is an equally critical factor.
Among the reasons for success, they also seemed to be factors that would make any team successful, regardless of if they are adopting Agile or something else. Many respondents described team empowerment and autonomy as a major factor, i.e. teams that are able to decide how they want to work together and create their own process as they go. Another major pattern was developers seeing the value in UX, or the UX role straddling the developer-designer divide (e.g. a UX designer who also is a front-end dev) were a strong pattern among those who described themselves as successful.
These responses re. the reasons for success seemed to capture the essence of what many were saying:
“For our teams, the UX designer is a dedicated team member – reporting to their product teams. Complete integration with a development team creates a much more successful team dynamic, shared ownership, opportunities for learning and growth, etc, in comparison to having UX as an external team/department.
We have also abandoned the idea that you always should design a sprint ahead, always use certain design methods, etc… Over the years, we’ve learned that there’s not “one right way” to do design & development. Some problems need the extra time to experiment and iterate and others don’t. Some designs need prototypes first, some can be iterated on in tandem with development. We’ve become far more flexible in our approach and techniques, over the years. We learn as we go.” — Alan Dennis
“Everyone talks about how important it is to keep the UX design ahead of the development sprints. I’ve found that this often means that the UX designer is the first to encounter tricky architectural issues. So one thing I’ve learned is that the entire team needs participate in the evolution of the UX design for a given feature or new area. This means that the development team must devote some % of their hours in a given sprint toward reviewing UX design for a future sprint and you have to factor this into the schedules. But if done right, both the UE and any important architectural coding issues are well understood before the story is picked up for implementation. Doing so can help avoid unexpected surprises that cause churn in the UI, features or schedule.” — Robin Silberling
“The UX designer and the Web Develper are part of the sprint planning and we sit next to the back-end Developers. The back-end Developers are very interested in watching the videos of the usability tests. Also they find the wireframes and Agile Acceptance Tests usuful when coding up the windows. We have had quick meetings with the UX designer, Web Developer and the back-end Developer to clarify what needs to be completed before coding starts. All developers are encourage to ask questions and give feedback on the wireframes or any part of the design/coding process. Before introducing the Agile Accpetance Tests the Developers were asked what they thought and if they had any feedback.” — Anonymous
Looking at the Numbers
A key focus here is on activities and process. We–rightly so, in hindsight–got a lot of feedback that we were too process-focused with this survey and were looking at UX as a separate person rather than a literacy. We’ll be working to correct that in future surveys. With that in mind, these were some findings relating to how UX is integrated into Agile teams.
UX-Designer Co-Located with Developers
Nearly 50% of respondents said that the UX designer is co-located with developers all the time. Once we start filtering findings, we’ll check this against those which identified as finding their agile adoption a success.
UX-Designer Doing Front-End Development
50% of respondents identified the UX designer as either regularly (20%) or occasionally (30%) also doing front-end development. Cross-disciplinary roles, i.e. team members wearing many hats is a strong indicator of successful/healthy team (see written responses below.)
Designing Using the Production Technology
45% said they regularly design directly in the browser and only 14% said they had no idea what that meant. If a team is designing in the browser, this is a strong indication that they have a tight integration between UX and dev practices.
The Last Artifact Created Before Development Begins
Detailed wireframes and functional specs continue to hold the lead, with 43% using this as what is the basis for initial development work, with hi-fi prototypes a close second at 33% percent. Only 15% go directly from sketches and whiteboarding to development. This is an area where there still is a lot of room for improvement. In my experience, UX designers continue to struggle with letting go of the deliverables mentality, the idea of UX being one of creating pretty-looking design artifacts before starting to create software.
When Does Detailed Design Work Occur Relative to Development?
Only 8% complete all detailed design work before any development work begins. 50% complete a majority before development. 35% are doing detailed design work in tandem with development. Here again, there is plenty of room for improvement. The idea of not designing everything in atomic detail before starting to build can seem scary to a traditional UX designer. There are probably also a lot of business/org culture/peer pressure factors here that will drive UX practitioners to feel compelled to create all this detailed design up-front.
More Written Responses
Reasons for Success
“UX designer is respected, full member of the team”
“We’re somewhat successful because I think previously we were working with high-fidelity mockups that made it hard to iterate in tandem with developers and design and implement an UX in an iteration or two. We’re now more focused on paper prototyping to give developers the guidance they need around implementing functionality, and designing more directly within the browser within the current iteration. This has been working better, but I think it’s taken a long time to get the hang of UX tools and methods that match the speed of development and iterative methods.”
“We explored the potential of every person & helped other person learn something new. In short we recognized the capability of the team. Made sure who the actual owner of the Story is & made others contribute in the mix. In short, we empowered the team to make decision & requested management solve the impediments & stay out of teams face.”
“The Tech lead is a huge supporter of UX and understands the value. And, if he ‘forgets’ about me at times, he does what needs to be done to get me integrated back in the dev cycle.”
“…the UX desinger has always been considered part of the team. I used to be a FE dev so understand the complexities. Am also a ScrumMaster and attended the training with the developers. Started from the same point. This means I’ve always had very good relationships & high levels of trust with the devs. Always get dev feedback on feasibility and we can all be part of the process.”
“Even though we’re geographically dispersed, we are constantly in touch via an ever-present group skype chat.”
Reasons for Lack of Success
“We have off-shore team, hence bringing them up-to-speed & style is getting challenging. Team here in US still has the mentality that US team is the best.”
“Too many executive stakeholders who don’t understand the value of agile methods. UX team (our client) still focused on producing deliverables rather than working code (site maps, content inventories, requirements, specs)”
“The organization started pure agile and is struggling to integrate ux”
“My organization still views agile and ux as diametrically opposed interests. They still view ux as an upfront activity. My local project teams have a changing view because of their experience working with me, and I hope that view spreads throughout the organization.”
“Front end developers requiring more design detail thus pushing more work onto design.”
“We don’t involve users. We create interfaces that are very flashy, but not terribly useful to the user.”
“The lack of co-location and general “design” think is the biggest barrier.”