At work we evaluated the fitness of one large collaboration platform for use for another group. The government was already funding this one big thing and it made sense to see if it could be leveraged instead of starting from scratch even though the potential user groups are extremely different.
The system we evaluated was carefully designed with lots of input from user groups, by well meaning, competent people, using best practices from the field. GAO has fussed at them a few times over the years for the same things they always pick up on and there are always questions about if their system is used enough and how and what contracts they have let and for how much. They have a roadmap for development that is carefully developed in coordination with the users and they use agile development with frequent small releases and quarterly larger releases. There's lots of training available both ad hoc, recorded, and live as well as in person presentations at conferences and the like. They have a bunch of case studies in which the system has had a pivotal role in supporting collaboration and solving a difficult problem for the users.
Sounds great, right? The only thing is that the actual system is pretty ugly and not all that functional - certainly not what we had been designing with our ambitious state of the art system. We asked about things like how access control is done, how information is organized and retrieved, how content management is done, what the portal does, how it supports communication and collaboration... all fell very far short of our expectations. How could this be? We were looking at current features in products on the market - we even looked at products they have.
In my opinion (not anyone else's), it's all about their users and their governance. They have proposed many of the things we want in our system and their users de-prioritize all of them and do not chose to fund them. You see, a lot is needed for really good content discovery - there's a lot of infrastructure, which is invisible to the user (see Star's stuff on infrastructure). There's a lot of humans developing and training information organization schemes and building ways to ingest and process information such that search works. There are the policy requirements in a federated system like this to allow these various repositories to be searched. There's ongoing maintenance and user testing and ranking and boosting and troubleshooting for even a decent search to work, not to mention the full content discovery.
So the professionals propose projects to work on these things and improve them, but the users - who are expert in an ENTIRELY different area - are not getting it and are not trusting the professionals. And money is always limited. So the communication pieces aren't integrated. There's not fine role based access control. There's no way to search across various things... But their users are happy and are getting EXACTLY what they asked for.
So how do you design a governance system and development for a massive collaboration system such that it is user-based and need-based, but you still can fund infrastructure work needed to provide the functions for the users. I don't know. We laid off our taxonomist because management thought our search tool did all that itself - it doesn't. Clearly we don't know how to make the case, either.
Is there hope? If the two systems are joined, might the developers leverage our information to force some of these improvements? Dunno.