Diagnosis and treatment of dementia
European Collaboration on Dementia
The aim of the EUROCODE project is to raise the profile of dementia within the EU Commission. This is being achieved through a series of EU Commission-funded syntheses of evidence and opinion. The project consists of five work packages to produce guidelines on drug treatment and diagnosis, psychosocial interventions, economics, epidemiology and prevention. This document is the output of the first of these. Recently, the Commission has agreed to develop an Alzheimer’s Action Plan. The results of EUROCODE have the potential to inform this Plan.
There is great variation in clinical practice and policy relating to dementia across Europe. This variation can leave policy makers unsure as to which clinical practices to fund and encourage. Numerous guidelines for the diagnosis and treatment of dementia have already been produced. Some of these documents include exhaustive analyses of the research literature. National guidelines frequently differ in the nuances of recommendations and occasionally, guidelines directly contradict each other.
The specific brief of this project was therefore to use existing guidelines as a basis for the development of a ‘meta-guideline’ on treatment and diagnosis of dementia, suitable for use across Europe. This is consistent with the increasing recognition that adaptation of existing guidelines can sometimes be more appropriate than re-creation of guidelines de novo.
The WP5 working group
This work package was hosted by the Cochrane Dementia and Cognitive Improvement Group, based in Oxford UK and led by its Coordinating Editor Dr Rupert McShane, who is an old age psychiatrist.
The other 5 members of the work package were Sigurd Sparr (Norway, Alzheimer Europe, Geriatrician), Mario Fioravanti (Italy, experience with International Psychogeriatric Association, Psychology), Peter Tariska (Hungary, dementia specialist), Philip Scheltens (Netherlands, neurologist, neuroimaging researcher), Brigitte Grass-Kapanke (Germany, psychologist).
During the course of the project, the group’s administrator Dymphna Hermanns moved on, and was replaced by Helen Collins. Amanda Kerr was appointed on the EUROCODE grant for 16 months, starting in April 2007. She remained working on the project for a further 4 months supported by Cochrane funds.
The starting point was to attempt a comprehensive collation of existing guidelines. This was not as simple as it might appear. First, the time frame of this project has coincided with a proliferation of new guideline creating organisations and also the maturing of some better established organisations. Second, the high level of interest in dementia as a public health problem in member states has meant that, during the timeframe of this project, there has been a continuing stream of new guidelines emerging. Third, guidelines which are the product of governmental departments are not usually published in conventional medical literature. They are therefore not accessible to the usual search techniques used by organisations which specialise in systematic reviewing of primary literature such as Cochrane.
Our cut-off point for identifying the existing guidelines which we would formally consider in developing our own was 1st quarter 2008. However, as we have become aware of new guidelines since then, we have read those too. There were no instances however, where a recommendation in the draft was altered as a result of the newer guideline since those which were identified after that point did not either add important new weight to existing controversies, or introduce new material which had not already been identified through review of the primary and secondary literature as worthy of inclusion in our guideline.
Guidelines were ‘includable’ if they covered aspects of the drug treatment or diagnosis of dementia. We decided to limit the guideline to common neurodegenerative dementias, thus excluding dementia associated with alcohol, HIV or Huntington’s.
Guidelines were includable if they originated from a European state after 1995. Older guidelines were ignored if superseded by more recent versions. The guidelines had to be produced by government agencies, professional associations, or the voluntary sector. We did not include guidelines with a sub-national or regional remit. Recent guidelines from outside Europe were also included on the grounds that these may contribute usefully, particularly when they addressed newer findings and controversies. We restricted consideration of non-European guidelines to those in English. We also considered new literature published since 2005 which addressed emerging clinical issues or provided important new evidence to guide treatment or diagnosis but had not yet been incorporated into any existing guidelines. We excluded consideration of cognitive enhancers (which are invariably short lived ‘boosters’ of attention) or analysis of drugs with potential utility for primary prevention because prevention of dementia is the subject of another work package. However, secondary prevention of progression from minimal cognitive impairment to dementia was in scope.
We decided to ignore the differences in GDP of the countries for whom the guideline was intended not least because variations are often as great within member states as they are between them. Thus the strength of recommendations should apply across the board. However, a guiding principle as to what was includable was that treatments or diagnostic techniques should be feasible for use in routine practice not just in tertiary research centres of excellence.
Methodological developments in guideline development
The period of this work has coincided not only with the release of many new guidelines, but also with significant developments in the technology of guideline creation. The governmental organisations which develop guidelines necessarily, as part of their mission, elaborate standard procedures for developing guidelines. Inevitably, this has meant that guidelines are produced in different forms. It is a recognised problem that there is no consensus on the way in which evidence should be graded. Similarly, there is no consensus on a grading system which should be used to describe the strength given to recommendations. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach started to be developed in 2000 but only really started to gain significant traction 2005, the year before our project began. As a method, however, it depends on assessing primary evidence, and this was clearly beyond the scope of our project.
The Adapte Framework
A very relevant framework for our project was launched too late for us to make formal use of its provisions. The Adapte Collaboration developed from two organisations, one French (Adapte) and one Canadian (Practice Guideline Evaluation and Adaptation Cycle PGEAC). The framework is a systematic approach to aid in the adaptation of guidelines produced in one setting to be used in a different cultural and/or organizational context. It is based on the following core principles:
- Respect of evidence-based principles for guideline development;
- Reliable and consistent methods to ensure the quality of the adapted guideline;
- Participation of key stakeholders to foster acceptance and ownership of the adapted guideline;
- Explicit consideration of context to ensure relevance for local practice and policies;
- Transparent reporting to promote confidence in the recommendations of the adapted guideline;
- Flexible format to accommodate specific needs and circumstances;
- Respect for and acknowledgement of source guidelines material.
Its founder members were oncologists and clinicians interested in the treatment of leg ulcers. The large majority of members are still from these countries and disciplines with some input from the Netherlands. The manual for guideline adaptation and evaluation study were launched in April 2007.
Potential advantages of ADAPTE
This would potentially have delivered a greater form of transparency to our decision making. It would also have had the benefit of giving a formal label to the process we used. In turn, this may have made a small difference to the credibility of the guideline.
Potential disadvantages of ADAPTE
Transparency of approach is core to the mission of Cochrane. It is relatively easy to deliver in the context of meta-analysis of primary evidence. Most of the steps in the ADAPTE framework have a rather clear way of ensuring that what has been done is open to scrutiny. However, there are is a crucial point in the framework where the basis for decisions requires judgement. This is at the point of detailed consideration of adaptation of recommendations. Whilst the toolkit suggests that justifications of any modification to the recommendations of the preferred existing guidelines are written down, it is far from certain that this commendable emphasis on documentation as part of transparency would actually make any difference to accountability. Ultimately, such modifications are a matter of judgement. What really matters is the extent to which they are either evidence based or, as is more usually the case in the absence of adequate evidence, they reflect the experience of the majority.
The ADAPTE method involves selecting no more than a few existing guidelines to ‘ADAPTE’. This would have been problematic. It would not have been constructive to select just one or two existing guidelines for adaptation because it would have led to the potential rejection of our work on the basis that it was not fully representative
Three of the 6 criteria for assessing the acceptability and applicability of recommendations were not fit for our purpose because of the large number of countries for which our guideline was intended
- The recommendation is compatible with the culture and values in the setting where it is to be used.
- There are no constraints, legislation, policies, or resources in the health care setting of use that would impede the implementation of the recommendation
- The intervention/equipment is available in the context of use
The methodology of the ‘consensus generating’ part of guideline development is not addressed in any detail. Whilst the ADAPTE framework is specifically designed for settings in which resources are limited, most consensus generating methods require substantial resources.
The Adapte framework is relatively new. Although it has excellent face validity, it has not yet been formally evaluated. Indeed, as mentioned above, the manual and toolkit were only launched in April 2007. We did not in fact became aware of its existence until 3rd quarter of 2008 which was too late for us to formally adopt and complete the manualised process.
Method used to develop EUROCODE guideline
This account and critique of ADAPTE is given because it helps to illustrate the arguments which led us to the rather simple, lean method we used to generate our draft guideline. This can be summarised as follows:
- Identify guidelines published before 1st quarter 2008.
- Translate relevant sections
- Cut and paste the text of each guideline’s recommendation about a particular issue into a table
- Load guideline into new guideline comparison website: http://guidelines.dementia-in-europe.eu/
- Researcher drafts provisional distillation of source recommendations
- WP leader drafts complete new guideline which seems to best encapsulate the texts drawn from all the parent guidelines. Make judgements to resolve differences between source guidelines on the basis of:
- Quality of source guidelines
- Quality of evidence
- New evidence since guideline was developed
- Clinical judgement
- Assign a rating of the ‘degree of clinical confidence’ in the draft recommendation, based on a judgement which combines
- Strength of evidence as ascertained in parent guidelines
- Strength of recommendations in parent guidelines
- Unanimity of recommendations between parent guidelines
- Working Group ratify draft guideline
The Search Method, Results and Guideline comparison are described in Appendices 1-3.
A critique of our method, and recommendations on whether and how to generate consensus on the document are made below.
Discussion of Method
Going beyond the evidence
The field of dementia research is, compared to some, relatively young. The number of records of randomised controlled trials in dementia which is held on the Cochrane Collaborations ‘Central’ database is smaller than many other clinical areas. The quality of these trials is routinely criticised. However, we are where we are. Rather than simply overlook areas of clinical practice because there is no evidence to inform it, the Working Group took the view that it would be better to fill in some of the gaps that exist in other dementia guidelines. This requires a degree of clinical judgement. An example of this is the approach to the drug treatment of behavioural and psychological symptoms of dementia. Clinicians require an algorithm which goes beyond first line treatment, and which, for example, differentiates between different types of agitation. There is no attempt in any existing guideline to present this in a comprehensive fashion, but we used our clinical experience to interpret the nuances of differences between guidelines to create an algorithm for first line, second line, third line… seventh line treatments. In this way, differing recommendations from source guidelines were combined in a way which seemed to us to make clinical sense. Although the justification for this would lie in whether application of this algorithm results in a better outcome than ‘care as usual’, it is rather unlikely that a clinical trial to assess this would be feasible, not least because, for many, the algorithm will be close to ‘care as usual’.
Different ratings in source documents
Some of the guidelines we identified arose from exhaustive reviews of the evidence. The differing systems for rating of the evidence (see Appendix 4) means that selecting the wording of recommendations from more than one guideline would have meant selecting recommendations with differing rating systems. The large number of permutations meant that trying to document a transparent way of ‘mapping’ every one of these onto a preferred rating scale would have been a drain on resources that would not ultimately have improved the quality of our guideline, and would have been of limited validity. We decided to use the approach adopted in the American Psychiatric Association guideline because, of all the various systems which are outlined in the appendix, its simplicity best reflects what clinicians usually need from a guideline:
[I] Recommended with substantial clinical confidence
[II] Recommended with moderate clinical confidence
[III] May be recommended on the basis of individual circumstances
As stated above, we did not pursue the final step in ‘validating’ the guideline. Here, we discuss the pros and cons of doing this, the various possible methods and make recommendations.
Our experience in identifying the large number of existing guidelines led us to carefully consider the merits of generating another fully validated one.
The tests of whether a further guideline would add value should be, in comparison with existing guidelines relate to process and to content:
Would this new guideline be more likely to change practice?
Would this new guideline be referred to more?
Would this new guideline be acted on more?
Are the recommendations more likely to improve outcomes?
Great effort and expense would be required to gather the evidence to answer these questions with any confidence.
Although the credibility of the source and familiarity with the guidelines do not ensure compliance, adherence to guidelines is higher when the recommendations are supported by evidence from randomized clinical trials. Guideline recommendations can lag behind appropriate changes in clinical practice because, by and large, secondary care clinicians stay up to date with the important primary evidence (1)
Therefore, an important role of guidelines is to educate trainees and non-specialists, and to consolidate clinician’s practice around the margins, rather than to change it fundamentally. Given this ‘consolidating’ effect, it is clearly important that harmful recommendations do not find their way into guidelines.
A further implication is that more frequent revisions have the potential to make guidelines more useful(1).
The presence of a controversial recommendation in a guideline can be a useful spur to research which ultimately takes the field forward. This is particularly justifiable if the recommendation reflects common practice. The appropriate degree of confidence should be ascribed to such recommendation. It may therefore be better for the guideline to be rich in recommendations with low degree of clinical confidence, rather than to leave areas of common practice without comment.
This is the line we took.
Guidelines therefore have two competing priorities
- Do not make recommendations which consolidate harmful practice
- Include recommendations with a low (albeit specified) degree of confidence
The only way of resolving these lies in a validation process which depends BOTH on an understanding of the evidence and also experience.
There are two conventional approaches to securing guideline ‘validation’. The first is to send it out to preselected experts for comment. The second is to have a ‘consensus conference’ to which a large number (say 20-200) people meet together for a day or two to consider the text. Typically, the first approach is used more by government agencies where the draft was generated by experts in Evidence Based Medicine but not the subject, and the second approach is used by professional organisations who are expert in the subject – and may also be experienced in Evidenced Based Medicine.
Sending a guideline out to experts for comment is perhaps more appropriate for updates. However, no individual can be familiar with the details of literature across the whole field. Experts will therefore concentrate on their particular area of expertise.
The Consensus meeting has the advantages that face-to-face dialogue can resolve differences. To be effective, a consensus conference needs a large number of experts and clinicians. This requires long lead times, needs to be led by ‘names’, and to have the backing of influential organisations. It is expensive because it involves people flying in from across Europe usually staying overnight. The efficiency of this process is questionable, especially when existing guidelines form the starting point, as in EUROCODE, or when the purpose is to update a guideline. Furthermore, such conferences may also not always be representative of the end-users. Experts who are present at such conferences are likely to know the literature better than non-academics. In areas where there is no evidence, the anecdotal clinical experiences of non-academics may carry less weight than those of academics. The views of the writing committee carry considerable weight.
Web-based consensus generation: the route for EUROCODE WP5 validation?
Several factors led to the idea that the business of a ‘Consensus meeting’ could potentially be conducted more efficiently, and more transparently, through a web-based system. First, neither existing method of validation is entirely satisfactory from the point of transparency. In consensus meetings, those who are confident in their knowledge of the evidence may hold more sway than those who are more accurate in their understanding of it. There may be little time for those who are unsure about a new proposal to review evidence. Participants may feel under pressure to sign up even if there are elements with which they disagree. It was recognised that pressures from home institutions and economies of time mean that face to face meetings are increasingly less attractive and difficult to organise. The green imperative to minimise unnecessary flights was also recognised. In any event, we were not funded to host a consensus conference. However, we also felt uncomfortable with sending the document out to just a few experts.
Although tele- and web-based conferencing has, during the time frame of this project, become a more widely accepted feature of working life, it is not suitable for larger scale collaborations.
We were further emboldened to consider attempting a web-based system by reports of web-based Delphi consensus generation(2-6) which started to be published during the life time of this project.
We therefore undertook to develop a pilot of an innovative process which would enable a large number of participants to vote and comment on each of the draft recommendations.
The process was defined as follows. First, a panel of clinicians is invited. Panellists are encouraged to contact colleagues whom they think might be interested in participating. Because there are no physical limitations on the number of panellists, this snowballing, ‘viral’ approach could potentially get greater buy-in from a wider geographical constituency than is conventionally the case with consensus panels.
Panellists log on to the website. The draft guideline is divided into small segments each of which has up to 20 statements / recommendations.
For each segment, the clinician can approve it in its entirety, or reject it with or without a comment. The clinician can also agree or disagree with the rating (I, II, III as above) for the statement.
The comments made are then incorporated into a second draft, which is generated offline. Whilst the proposed method would be likely to generate a large number of comments, the fact that one would already have an idea of the extent to which a recommendation was acceptable in its current form would actually make it easier to decide how far to revise the text on the basis of such comments. This also provides a transparent mechanism for getting consensus on the issue of the ‘degree of clinical confidence’ that should be attached to a recommendation.
The clinicians would then be contacted again and invited to comment on the changes and to vote on changed recommendations.
Following this a third draft would be prepared. The clinicians would once again contacted and asked whether they were prepared to put their name to the document.
Throughout, panellists can additionally engage in discussion with each other through a web-based forum.
An innovative aspect of this, compared to the few other web-based Delphi guideline consensus generation processes is that the system is structured so that clinicians can access all the ‘parent’ guidelines and other quoted sources of evidence which lie behind each statement. These sources of evidence could include ‘summary of findings’ tables, such as has recently been adopted in Cochrane reviews.
Advantages of web-based consensus approach
‘Evidence is international, but guidelines are local’ (Andy Oxman, Cochrane guideline guru). This observation points to a potentially fundamental barrier to the EuroCoDe guideline gaining currency. However, as public finances become increasingly constrained, the capacity of Health Technology Assessment agencies, and professional assoications of different countries to update local guidelines may be reduced. A web-based system, if it demonstrated good ‘buy-in’ from large number of clinicians from each member state, might increase the appeal of EuroCoDe in individual member states.
The ‘viral’ approach to gathering consensus participants has the advantage that a large sample of clinicians can be rapidly built. Although it is possible that interest groups supporting a particular agenda may develop, this would be readily identified.
The linkage of each statement to the underlying primary evidence would result in an invaluable educational tool. Taken to its logical conclusion, it could become the structure for a ‘tree of knowledge’ for our speciality.
By de facto incorporation of many of the principles of the ADAPTE methodology, it has the potential for more rapid, transparent and efficient updating. In an era of rapidly advancing knowledge, it may therefore improve the currency of the guideline. This is likely to increase its utility, buy-in and impact.
We recognise that the structure we developed was not specific to dementia. Indeed, if this was successful, it could be applied as a methodology for gathering consensus on any guideline.
State of pilot at end of project
As far as we have been able to ascertain, this had not been attempted before. Its outcome was therefore uncertain. We therefore took the view that the Cochrane Dementia Group should contribute support for this pilot as a way of developing experience in web-based consensus development.
The software for the project has reached an advanced state of preparation and can be viewed at http://www.eurocodeconsensuspanelwp5.org/. It has yet to be populated with the full draft guideline and we have not run the proposed process. We have done much of the work in developing the structure of the site and work-flows but the final Proof of Concept work has not been completed.
We judged that the web-based consensus-development tool we developed, or something similar, has the potential to form the basis of a larger, more formal project. Such a project would
- establish consensus on the EuroCoDe guideline
- establish a structure for updating the guideline
- establish a peerless structure for linking evidence to a guideline
- establish a structure which could be a model for other disciplines
This larger, successor project would need to formally involve people with the wider expertise and experience, including
- Guideline development – preferably to include people from outside the field of dementia
- Web-based Delphi consensus generation
- Business analysis
- Project management
- Software development
- Clinical expertise
- Evidence based medicine
Last Updated: Thursday 13 August 2009