That s a pretty general question - you probably need to be more specific. However, here are some starting points that you might be able to think about.
Cost of user support - number of user support incidents as a proportion of the user base. Break it down by task, module or some other breakdown that you can link to actual application functionality that could be fixed. Bonus if you can get stats on the rate of effective fixes.
Cost of data fixes - depending on your application, cost of having to fix incorrect data may be a support cost metric that you should track.
Number of mistakes - if you can spot erroneous navigation patterns in web server logs you could try to bring the level of incidence of this down.
Retention or number of aborted attempts. If you have a public web site then you might want to spot the incidence of users losing interest.
These are all questions that you are quite likely to be able to get hard data on (at least for a web application) and they will give you some objective measures that are correlated to real aspects of performance. Notice the correlation to costs and effectiveness (punters leaving the site, cost of support calls). From naive economic perspective these will give you basic quantitative measures about how effective the site is at doing its job (I m assuming you re working for someone maintaining a web application).
You can probably think of similar measures that are more directly appropriate to your application.
To get at a deeper view, you may want to do some traditional marketing/HCI stuff like surveys or even focus groups or usability testing. Beyond that you are evaluating the underlying strategy of the system which (I guess) is probably starting to fall outside your brief. Start with things that you can measure first.
