Ewa Dobrogowska-Schlebusch, Barbara Niedźwiedzka...
Transcript of Ewa Dobrogowska-Schlebusch, Barbara Niedźwiedzka...
Ewa Dobrogowska-Schlebusch, Barbara Niedźwiedzka
Department of Information Studies. Institute of Public Health, Jagiellonian
University, Kraków. Poland
Nearly every second person
participating in the study was
shown to have limited
(inadequate or problematic
health literacy)
It is estimated, that almost
40% of Polish citizens have
poor health literacy level
To create evidence-based,
comprehensive, easy to find
source of health information
for Polish citizens
Department of Information Studies of
the Institute of Public Health
(Jagiellonian University, Kraków) is
currently conducting the study:
„Assessment of the quality of online
health resources in order to identify
the examples of best practises in
creating portals for patients”.
Main aim of the study:
To identify health information websites
of best quality created for patients/
healthcare consumers which could
serve as models for future Polish
website(s)
How to evaluate existing
portals to find the best ones?
How many evaluation tools
exist? Which is the best?
Something that an Internet
user could use to assess the
quality of website containing
health information (Breckons et
al. 2008)
A literature review done prior to conducting
the research showed that although many
instruments used to rate the quality of health
information websites had been developed (in
2005 Bernstam et al. Identified 273
evaluation instruments) none of these seems
to have been validated
The question was: should we
validate one of the existing tools
or is it better to develop on their
base a new one, and validate it?
Source: pixbay.com
We decided to to the last step…..
What is actually the
aim of this
presentation?
Source: http://openclipart.org
source: frickr.com
To show the preliminary phase of
the project:
The development and validation of a
new tool for evaluation of health
information websites
(Narzędzie Ewaluacji Stron
Internetowych o Zdrowiu – NESIOZ).
source: http://pixabay.com
* I phase of the study: to compile set of quality
criteria most commonly used in instruments
developed to assess the quality of health
information websites
* II phase of the study: pilot test of NESIOZ,
version I with a group of 31 public health
students
* III phase of the study: pilot test of NESIOZ,
version II with a group of 22 dietetics students
* IV phase of the study: pilot test of NESIOZ,
version III with a group of 14 dietetics students
and a group of experts
Systematic review of available health information websites’ quality
assessment instruments was performed (PubMed Medline,
Scopus, Embase, ISI Web of Science, Academic Search Premiere)
from January 2008 through August 2013
Instruments were eligible for review if they had been used at least
3 times in the research in the past five years to evaluate the
quality of health information websites with their rating criteria
available
Each rating instrument included to the study was examined on
quality criteria applied
Most frequently used quality criteria were extracted, analyzed and
later rewritten as questions
I version of NESIOZ tool was created
A total of 132 studies published in the last five
years, were found where the quality of health
information websites was assessed with quality
rating instrument
8 instruments were found, which had been used at
least 3 times to evaluate the quality of health
information website in the last 5 years
Rating instruments: How many times it was used to assess
the quality of websites in the last 5
years?
1. DISCERN 92
2. HON Code of Conduct 31
3. JAMA benchmark 23
4. LIDA tool (Minervation Validation
Instrument for Health Care Web Sites)
22
5. Khazaal scale 8
6. University of Michigan Website Evaluation
Checklist
5
7. Sandvik Score (general griteria) 4
8. EQIP 4
DISCERN
–a questionnaire developed
by Charnock et al. (1999)
which provides users with a
valid and reliable way of
assessing the quality of
written information on
treatment choices for a
health problem.
Currency of information (the original posting date is given; date of last update, frequency of update).
Attribution and documentation (presentation of clear references, details about additional sources of information, balanced/unbiased evidence, evidence-based
Authority of source (authorship, credentials/ qualifications of authors)
Disclosure of developers, sponsors (ownership): (identification of purpose, funding source/ sponsorship, advertising policy)
Contact addresses or feedback mechanism (availability of contact information/ possibilies of sending queries to a postmaster, availability of support/ help section, interactivity/ chat/ feedback forms/ supporting bodies)
Appropriateness for intended users
Quality/ relevancy of links
Design and aestetic
Ease of use (including navigation)
Accessibility and availability
Content of site (currency of information, attribution and
documentation)
Disclosure (authority of source, funding source including
advertising policy)
Design and aestetics (layout, navigation)
Availability and accessibility (fee for access/ registration,
interactivity, disclaimer)
User support (availability of contact information, site map)
Appropriateness for intended audience
The NESIOZ tool, version I
was pilot tested with a
group of 31 public health
students (Jagiellonian
University – Medical
College) in order to
determine:
the usability of the tool
the clarity of its question
to measure how much time is
needed to asses the webpage
with a tool
http://portal.abczdrowie.pl
Students taking part in pilot test mostly expressed positive opinions about utility of the tool
Pilot test group reported it took about 15 minutes to complete the evaluation process
The largest differences in students’ evaluation emerged in regard to questions pertaining to: the date of last update, authority of source, disclosure of financial support, advertising policy and ease of use
These questions after pilot test I
were rewritten in order to increase
their clarity.
The NESIOZ tool, version II
was tested with a group of 22
Dietetics students to further
determine the clarity of its
questions and instructions.
The students were required
to evaluate the quality of 5
selected nutrition websites:
www.apetyt-na-zdrowie.pl,
www.eufic.org, www.mojacukrzyca.org,
www.dieta.mp.pl,
www.sztukaodzywiania.org
The greatest discrepancies between
students’ evaluations were found in
questions pertaining to design and
web site aesthetics (Is the website
layout clear?), ease of use (Is the
website easy to navigate?) and
advertising policy (Is advertising
distinct from content?)
terms and condition of website
Main changes:
* One question (concerning terms & conditions of website use)
was deleted from a tool
* Some revisions were made in order to further increase the clarity of questions (e.g.
guidelines/ principles of good website layout and navigation techniques were
provided)
NESIOZ tool, version III is currently being tested with
a group of students and experts (medical librarians
or information science scientists) to establish its
reliability and face validity.
Especially experts’ opinion will be taken into
consideration to further improve the utility of the tool
It is impossible to achieve consensus over website’s quality assessment as assessing some aspects of website (the aesthetic visual quality of website, ease of navigation, advertising distinct from content) is a highly subjective task and determined by personal characteristic of the evaluator.
Despite the presence of relatively precise definitions some criteria remained difficult for evaluators to assess (Sagaram et al. 2004)
The criteria used must be unambiguous so that the different people applying them will agree upon their usage (Sagaram, et al.)
The problem is probably to large extent not the tool itself, but the websites which were the subject of assessment in the pilot tests
The assessment of health information website even with a very short and precise evaluation tool might pose a problem for the untrained user
Thank you very
much for your
attention!
Source: http://en.wikipedia.org/wiki/Bowing