Graphing a Biomarker-Driven Research Program
One of the challenges in biomarker research is that there is often no single stakeholder that owns the biomarker and is capable of coordinating the research program. This can lead to duplicative studies that are addressing the same question or low-value studies that continue to test a biomarker hypothesis that has already been refuted. Since every investigation involving human subjects (or their specimens) is obligated to redeem its costs and burdens by making a significant contribution to scientific knowledge, these sorts of research inefficiencies are ethically problematic.
In an effort to measure, and ultimately reduce, the number of duplicative or low-value studies, the PORTAL Biomarker Research Consortium is using Accumulating Evidence and Research Organization (AERO) graphing to depict the portfolios of studies that make up a biomarker-driven research program. The basic idea behind AERO graphing is to provide a systematic method for representing research programs, and in this way, facilitate greater transparency, communication, and coordination among regulators, funders, investigators, ethics committees, and participants about the context and value of every study.
In an effort to measure, and ultimately reduce, the number of duplicative or low-value studies, the PORTAL Biomarker Research Consortium is using Accumulating Evidence and Research Organization (AERO) graphing to depict the portfolios of studies that make up a biomarker-driven research program. The basic idea behind AERO graphing is to provide a systematic method for representing research programs, and in this way, facilitate greater transparency, communication, and coordination among regulators, funders, investigators, ethics committees, and participants about the context and value of every study.
CETP inhibitors and the HDL hypothesis
We recently published an analysis of one biomarker-drive research program—the cholesteryl ester transfer protein inhibitors (CETPi) as treatments for cardiovascular disease—using the AERO approach. Development of this class is driven by the hypothesis that it will raise HDL cholesterol, lower LDL cholesterol, and thereby reduce the risk of cardiovascular disease. The published paper includes a static AERO graph and detailed discussion of the CETPi portfolio. Below is an interactive graph of the CETP trials.
However, since that paper was published, more study results have become available (and some significant business decision have been made). This interactive AERO graph can go beyond the published paper to take those new results into account. It also allows for a more in-depth exploration of what we found. For example, you can click on nodes to open the published report (in a new browser tab) or the NCT record (if the study has not been published). You can also zoom in and out (using the tools on the right) or just mouse over nodes to get more information about each study.
However, since that paper was published, more study results have become available (and some significant business decision have been made). This interactive AERO graph can go beyond the published paper to take those new results into account. It also allows for a more in-depth exploration of what we found. For example, you can click on nodes to open the published report (in a new browser tab) or the NCT record (if the study has not been published). You can also zoom in and out (using the tools on the right) or just mouse over nodes to get more information about each study.
What is going on in the graph?
Every node in this figure corresponds to a study involving human subjects. The nodes are arranged by date of publication along the x-axis (or study completion date if there is no publication) and stratified by drug and primary endpoint along the y-axis. The color of the node corresponds to the study’s result. If a study found a positive result on its primary endpoint, it is colored green; a negative result or terminated study is red; a mixed or ambiguous results is yellow; an unpublished study whose outcome is unknown is grey; and an on-going study is blue. The size of the node corresponds to the number of human participants enrolled.
How is this graph helpful?
For regulators, who have perhaps the widest perspective on the research enterprise, these graphs could be used to communicate how the regulatory agency views the total state of evidence for or against a particular biomarker hypothesis, and the type of trial that would be required to sufficiently demonstrate an experimental drug's safety and efficacy in light of that evidence. Drug developers and investigators could then take this information into account when planning future studies.
For funders, who also have a wide perspective on the research enterprise, this approach can help to structure thinking about the total portfolio of work they support. If the AERO graph reveals that some hypotheses, interventions, or study types have been overlooked, funding agencies could put out a call for proposals or prioritize applications that would fill those gaps in our knowledge. Alternatively, if the graph reveals that a hypothesis has hit a point of diminishing returns, funders could prioritize work in other areas.
For investigators, this approach can similarly help them to analyze the current state of knowledge and activity in a given area, and then to plan their next study accordingly. Once an investigator has identified an important question and designed their trial, they can then use the graph to communicate with the other research stakeholders about the existing body of evidence, the known risks, and the likely positive social and scientific value that their study will contribute.
For ethics committees, AERO graphs elucidate how a particular trial protocol fits into its larger research context, and can thereby help the committee to think systematically about the total risk/benefit balance. For example, if the committee is reviewing a protocol and sees that there are several other studies addressing a similar question, they could ask the investigator to elaborate on how the proposed study differs. Or if previous similar studies have failed to accrue sufficient participants or uncovered safety signals, these issues can also be raised with the investigators to clarify plans for adequate recruitment and monitoring for safety events.
Finally, for potential trial participants, the graph can help them decide whether or not they want to enroll and accept the risks and burdens of a study. Indeed, truly informed consent would require that a participant has knowledge of the prior research surrounding and leading up to a study, as well as knowledge of any alternative studies that might be more suitable for them. This interactive representation of a research program complements a narrative review of the existing evidence, allows patients to explore past and concurrent trials, and can serve as a touchpoint for discussion between a patient and physician about whether a particular trial is right for them.
For funders, who also have a wide perspective on the research enterprise, this approach can help to structure thinking about the total portfolio of work they support. If the AERO graph reveals that some hypotheses, interventions, or study types have been overlooked, funding agencies could put out a call for proposals or prioritize applications that would fill those gaps in our knowledge. Alternatively, if the graph reveals that a hypothesis has hit a point of diminishing returns, funders could prioritize work in other areas.
For investigators, this approach can similarly help them to analyze the current state of knowledge and activity in a given area, and then to plan their next study accordingly. Once an investigator has identified an important question and designed their trial, they can then use the graph to communicate with the other research stakeholders about the existing body of evidence, the known risks, and the likely positive social and scientific value that their study will contribute.
For ethics committees, AERO graphs elucidate how a particular trial protocol fits into its larger research context, and can thereby help the committee to think systematically about the total risk/benefit balance. For example, if the committee is reviewing a protocol and sees that there are several other studies addressing a similar question, they could ask the investigator to elaborate on how the proposed study differs. Or if previous similar studies have failed to accrue sufficient participants or uncovered safety signals, these issues can also be raised with the investigators to clarify plans for adequate recruitment and monitoring for safety events.
Finally, for potential trial participants, the graph can help them decide whether or not they want to enroll and accept the risks and burdens of a study. Indeed, truly informed consent would require that a participant has knowledge of the prior research surrounding and leading up to a study, as well as knowledge of any alternative studies that might be more suitable for them. This interactive representation of a research program complements a narrative review of the existing evidence, allows patients to explore past and concurrent trials, and can serve as a touchpoint for discussion between a patient and physician about whether a particular trial is right for them.
For more information
Part of the goals of the PBRC is to develop a set of user-friendly tools that would allow research stakeholders to easily construct and share these graphs. While we have made use of this approach to elucidate issues in biomarker research policy, we believe that the real power of the approach lies in the prospective use by researchers themselves—to help plan and coordinate their work.
If you are a researcher or policymaker interested in constructing AERO graphs or a programmer interested in working with us on developing tools to help others do so, please e-mail Spencer Hey ([email protected]).
If you are a researcher or policymaker interested in constructing AERO graphs or a programmer interested in working with us on developing tools to help others do so, please e-mail Spencer Hey ([email protected]).
