General comments
Three broad issues need to be considered when appraising an information needs analysis:
A/ To what extent is the study a close representation of the "truth" (validity)?
B/ Are the results credible and repeatable (reliability)?
C/ Will the results help me in my own information practice (applicability)?
1. Does the study address a clearly focused issue? |
|
||||
|
|
|
||||
|
HINT: An issue can be "focused" in terms of : · The population (potential user group) studied · The definition used for "information" · The methods for assigning preferences or weightings |
Yes |
Can't Tell |
No |
||
|
|
|||||
|
|
|
||||
2. Does the study position itself in the context of other studies? |
|
||||
|
HINT: Consider whether the study: · Refers to other studies that have looked at the same user group · Utilises a methodology that has been used in previous user studies |
Yes |
Can't Tell |
No |
||
|
|
|||||
|
|
|||||
3. Is there a direct comparison that provides an additional frame of reference? |
|
||||
|
HINT: This may either be external
: |
e.g. contrast with, or
similarity to, other studies |
Yes |
Can't Tell |
No |
|
|
Or internal: |
e.g. contrast with, or similarity to, other user groups within the study e.g. contrast with, or similarity to, the same group at different geographical locations or at a different time period. |
||||
Is it worth continuing? |
|
||||
|
|||||
Detailed Questions |
|||
4. Were those involved in collection of data also involved in delivering a service to the user group[1]? |
|||
|
HINT : It may not always be possible to separate researchers from
service deliverers but consider has the service deliverers' perspective been
acknowledged explicitly and to what extent have the questions in the user
study been generated elsewhere (e.g. a previously trialled or validated
instrument or from a focus group) |
Yes |
Can't Tell |
No |
|
|
|||
5. Were the methods used in acquiring data on information needs appropriate and clearly described? |
|||
|
HINT: Has a variety of written and oral, formal and informal sources
been used? Has an attempt been made to triangulate findings across a range of
sources? Has a range of methods been used to acquire data e.g
interview, focus group, questionnaire, observation etceteras? How have
respondents been identified? |
Yes |
Can't Tell |
No |
|
|
|||
6. Was the planned sample of users representative of all users (actual and eligible) who might be included in the study? |
|||
|
HINT: Do the demographics of the sample (e.g. age, sex, staff grade,
location) accurately reflect the demographics of the total population? Are
any interests or motivations behind participation clearly identified? Are
non-users included in the sampling frame? |
Yes |
Can't Tell |
No |
B/ Are the results credible and repeatable? |
|||
7. What was the response rate and how representative was it of the population under study? |
|||
|
HINT: Consider not only the actual percentage of responses but also
whether any specific subgroups were either over-represented or
under-represented. Are reasons for non-response discussed? Have non-users
been included in the analysis of responses?[2] |
|
||
8. Are the results complete and have they been analysed in an easily interpretable way? |
|||
|
HINT: Consider choices involved in analysis and in presentation. Have all variables identified earlier in the
paper been analysed[3]? If
not, why not? |
Yes |
Can't Tell |
No |
9. What attempts have been made to ensure reliability of responses? |
|||
|
HINT: Consider whether a single interviewer/team of interviewers was
used. Was training offered? What mechanisms were used to ensure consistency
of coding between interviewers? What relationship did interviewers have with
interviewees? What opportunities were there for interviewer bias? |
Yes |
Can't Tell |
No |
C. Will the results help me in my own information practice |
|||
10. Can the results be applied to your local population? |
|||
|
HINT: The burden of proof is on you to identify any ways in which
your local population might differ from that in the study. Are there
differences in the context
(political, environmental or organizational) of the study? |
Yes |
Can't Tell |
No |
|
|
|||
|
If
NO, are you able to use the same
methodology (if valid) with your local population? |
|||
|
|
Yes |
Can't Tell |
No |
11. What are the implications of the study for your practice? |
|||
|
In terms of current deployment of services?: |
|||
|
|
|||
|
In terms of cost?: |
|||
|
|
|||
|
In terms of the expectations or attitudes of your
users?: |
|||
|
|
|||
12. What additional information do you need to obtain locally to assist you in responding to the findings of this study? |
|||
|
|
|||