Proposed Interoperability Standards Measurement Framework Public Comments

Dear Sir or Madam,

This measurement framework is a good talking point and educational tool to help separate some of the issues. A few thoughts from an early stage Health IT vendor perspective:

1) As an early stage digital health company, we would noly integrate with an EHR based on customer request and need only. We do not consider this to be a true a "standards-based" integration. For any integration be useful, it would typically be a custom integration involving the partnership of the customer's IT leadership.

2) As an early stage company, we would consider attestation to standards at this time to be an impractical, enormous regulatory burden prior to FHIR being fully built out (Fast Health Interoperability Resources).

3) We respectfully request that any vendor-facing activity or regulation is enforced only for the results intended. We want to discourage the tracking and reporting of anything that doesn't drive real results for interoperability.

Briefly, the way we perceive the challenges are: ThereÕs existing standards, HL7 v2 and C-CDA, (all pre FHIR) and they are not going away. People are going to continue to use those systems because they can exchange lab orders and lab results, between the EHR and the lab system. In addition, different parts of the FHIR standard for packaging discrete bundles of data are not yet built out, particularly those parts that deal with complex clinical data elements such as specialties and genetics. The complex clinical portions are where interoperability is needed to care for patients with multiple co-morbidities efficiently. As HL7 board member Dr. Russell Leftwich points out in Healthcare Informatics, "What needs to happen is that the entire FHIR community and the clinicians in particular need to get together, one data element at a time, and agree on a model, or how to represent that piece of data. You have to have that to really have interoperability across organizations."

We would encourage ONC to focus on how to make progress with FHIR as quickly as possible with the right incentives. Without FHIR, interoperability amongst different brands of different systems, interoperability between different systems will continue to remain as custom built solutions. Therefore, reporting and tracking from our perspective appears irrelevant. FHIR interfaces will take hours or days instead the weeks it takes to build HL7 interfaces. If there really is a drive to move the needle on interoperability, there needs to be incentives i.e. financially positive for this work to come to completion. Perhaps this could mean making a very large grant available for various conveners such as the CARIN alliance to apply for and organize and drive specific portions, or maybe it's grants to specialty societies (American Academy of Cardiology etc.) who as part of the grant have to pay and assemble the clinicians needed to build specialty and genetics FHIR elements. In our view, there would need to be monetary compensation made available to the clinicians who agree to take on the needed leadership roles to drive representation of data elements and the models. Clinicians already feel overburdened by the current documentation requirements. While clinicians' input is needed to drive interoperability, they should be fairly compensated for their time in our view.

Respectfully,

Sherri Douville
CEO, Medigram Inc.

Doing my first read through, I noticed I had several additional questions I wish this proposal addressed:
When you mention Health IT developers publicly reporting, I'm wondering where to? Additionally, if there is an annual date per year (or if just roughly once a year?)?
Lastly, where is the over site. The pile of data would be fantastic, and I agree knowing the standards of use would be incredibly helpful - but the data needs to be compile, organized and maintained. How do we do that and where? How do we publicly promote this data so that its widely known and available?
Lastly, I was wondering how we encourage industries which are falling behind to speak up and present they're data fully and accurately. I feel like I can find plenty of success stories of large companies investing and succeeding, but how do we spot light the smaller firms, and whats more; support them? The US is covered in plenty of small companies that manage this on a daily basis, and they're data would be incredibly insightful.
1) I think if we're going to implement standards reporting, it has to be mandatory. Otherwise, only the industries and companies which can easily afford to will participate. In order to get the best possible data, from the most widespread sources, we need to implement this for everyone.
2) an annual survay or even a written progress report would be great. Something that not only tracks what software and updates are being used, but also how much, why and how that use is reflected in the interactions with the patient.
3) I think so? I really admire the goals, and think they are desperately needed.
4) I think we're lacking the overview; what we're doing with the end data, and how to interpret what it means.
5) I think personal injury firms, which not only aggregate data but communicate with both doctors and insurance companies; mediate care and manage payment of costs should be a voice in this discussion.
6)I think monitoring this data is something our industry sorely lacks, and while I think it is a huge task we are laying before ourselves, without it we will be charging forward without checking our math, so to speak. I think implementing this kind of standard will create more jobs and grow out field, too. We need to be a loud voice in the industry and checks and balances are a part of that.
7) I definitely believe it's feasible. At this point, many organizations require an annual review of they're performance. Including submitting data to ONC shouldn't be overly burdensome, especially considering we hope to streamline the process and provide stakeholders with resources on the overall effect of policies. I think the challenge may be in nailing down a date/deadline, since so many companies and industries operate differently.
8) Surveys on what standards are used would allow us to monitor and measure only the standards that are absolutely necessary. I think the idea of saying "its not possible to measure everything so we should pick and choose" will limit our data and make the end goal more difficult to determine. We need to know what the entire industry uses, how often, and why. I think we need a broad overview and then we can focus down on what we're actually using after analyzing the data.
9) I think creating a common vocabulary and definitions is part of this process. Would it be possible for ONC to create a vocab list? If the end goal is to great a guidebook, basically, then defining everything accurately so we can all use the same terminology should be the first step in the processes. If ONC is the body starting this processes, I think its feasible that they (you) establish the nomenclature.
10) I think this is the most complicated question asked, at least for me. I've read through the proposal twice and I think that tracking this while not violating the security we've worked to establish would be complicated. But in the end, I think we need some kind of tracking software that monitors use without retaining ePHI or any other confidential data. We need to know in what way software is used, even if we can't/shouldn't track the specifics that are transacted.

I am working in the HIT and interested about standards.

I look forward to providing feedback after review of the document.

Very good framework

Dear ONC stakeholders,
Thank you for providing us with the opportunity to share comments and feedback on your proposed model. No one can argue against the need and the value of measuring the adoption of interoperability standards. Measuring our progress is a very important activity in the journey toward universal interoperability.

I would offer the following observations:
1) The industry would benefit from a more comprehensive definition of interoperability standards and/or standard levels. Although most stakeholders will refer to Òinteroperability standardsÓ as a set of data values associated to defined set of data elements, it can also refer to the format used to express these values as well as the standard protocols used to exchange or transport the values.
2) There is limited value in measuring health system or vendor specific standards; the only real way to measure national interoperability progress is to measure the level of adoption of nationally defined and adopted standards (ICD-10 codes, medications and NPIs being the obvious examples). For example, there is limited value nationally in knowing that health systems have standardized on their own patient identifiers, order-sets, or their own list of gender identifiers.
3) The ultimate interoperability limiting factor is the non-standardized, sometimes unstructured, and rapidly evolving nature of health care data. Most health systems have to process non-only Òcurrent dataÓ, which may comply with current standards, but they have to process historical data, potentially based on different standards. Then they have to deal with new types of data being introduced by precision medicine technologies, by connected care (e.g. wearables and home monitoring devices) and other patient generated data.

To address these challenges, I would suggest that ONC considers the following recommendations:
- Limit the measures to standards that are well defined and generally accepted nationally and expand that set of standards over time.
- Push for automated reporting Ð as opposed to using surveys. As you mention, most vendors have the ability to monitor and report on the transactions. They also have meta-data tools that can be leveraged if the scope of the transactions is realistic and practical.
- Consider measuring which portion of the health care data set is actually covered by well-defined and nationally accepted standards Ð independent of adoption levels.
- Consider adding a ÒsharingÓ measure, i.e. monitoring how many health care providers can share a given patient record. With the consolidation taking place in the industry, and increasingly more stakeholders sharing platforms, national interoperability will greatly benefit from the increased level of sharing (as opposed to focusing only on exchanging). And in many cases (between hospitals and affiliated clinics for example), sharing the same platforms and patients is far more efficient than any interoperability solutions Ð that should be recognized and measure as well!

I hope this is helpful,

Carol

At this stage I am interested in reading the document so I don't currently have feedback to offer you.

Interoperability is critical to successfully achieving the Quadruple AIM and ensuring the success of value-based payment models. We appreciate the work of ONC to drive interoperability and we urge ONC to continue to prioritize this work. We are highly supportive of measuring interoperability throughout the health care ecosystem as we cannot improve what we do not measure. We also support a far greater level of transparency in the market place especially among health IT developers and health care systems. At the same time, we have ample concern that physicians and their practices will have a disproportionate share of the burden in measuring interoperability. We strongly urge ONC to not place more administrative burdens on physicians, as we already have significant burdens due to the lack of interoperability.

We appreciate the opportunity to provide ONC with our thoughts on some of the key questions laid out in the proposed interoperability standards measurement framework.

Introduction:
Interoperability is critical to successfully achieving the Quadruple AIM and ensuring the success of value-based payment models. We appreciate the work of ONC to drive interoperability and we urge ONC to continue to prioritize this work. We are highly supportive of measuring interoperability throughout the health care ecosystem as we cannot improve what we do not measure. We also support a far greater level of transparency in the market place especially among health IT developers and health care systems. At the same time, we have ample concern that physicians and their practices will have a disproportionate share of the burden in measuring interoperability. We strongly urge ONC to not place more administrative burdens on physicians, as we already have significant burdens due to the lack of interoperability.

We appreciate the opportunity to provide ONC with our thoughts on some of the key questions laid out in the proposed interoperability standards measurement framework.

1) Is a voluntary, industry-based measure reporting system the best means to implement this framework? What barriers might exist to a voluntary, industry-based measure reporting system, and what mechanisms or approaches could be considered to maximize this systemÕs value to stakeholders?
A voluntary reporting system would ensure that there is alignment with the value to those reporting, as if there is not value, they will not report. This of course means that the reporting system must provide more value than the effort required to report, which may limit reporting. We believe that an incentive based (voluntary) reporting is more likely to get us to the end desired state albeit at maybe a slower rate than a mandatory reporting system. Physicians are already over burdened with administrative requirements and lack of interoperability; let us not add additional mandated reporting burdens.
2) What other alternative mechanisms to reporting on the measurement framework should be considered (for example, ONC partnering with industry on an annual survey)?
A multi-faceted approach is most likely needed given the current state of interoperability and the industryÕs nascent ability to measure it. An annual survey would be a useful facet to gauge the general level of interoperability. We also believe that complaint or end-user issue reporting can also be useful to categorize the types of issues that are being encountered. This would help inform the construction of the annual survey. In addition, the complaint/issue reporting could also help surface industry entities that may be information blocking or having a specific issue with interoperability. Finally, an ability to audit implementations as such issues surface would be warranted to determine appropriate actions. This can be augmented with measures of transaction volume, both in volume of transactions and variety (message types and sender/receiver type) of transactions.
3) Does the proposed measurement framework include the correct set of objectives, goals, and measurement areas to inform progress on whether the technical requirements are in place to support interoperability?
If one uses the flow of information use as (1) data are made available for exchange, (2) they are exchanged, (3) they are received, (4) they are interpretable, and finally (5) they are usable, then the framework is incomplete to measure wide-scale interoperability. We have heard from many family physicians that are able to receive data into their practice, and the data are interpretable from the standpoint of showing the exchanged data on the screen (i.e. display a C-CDA with an XSL) but the data within the C-CDA is not usable from the standpoint of their EHR. The physician and/or staff are required to manually integrate the new data into the patient record. We would argue that if the exchanged information is not usable by the receiving health IT system to support integration into a patientÕs record, then we have yet to achieve interoperability; we have only achieve health information exchange.
4) What, if any gaps, exist in the proposed measurement framework?
As stated in the answer to question three, interoperability is only achieved when the exchanged information is usable by the receiving system and ultimately the user. We have concerns that a gap exists in the current proposed framework, as the usability of the exchanged information is not addressed. Additionally, there is a gap in the measurement of the impact of the standard and its implementation on usability.

We would urge ONC to consider adding a component to the framework that expressly deals with the usability of standards-based exchange of information. ONC should consider a methodology including focus groups of clinicians (and other end-users as appropriate) to qualitatively understand the usability issues around standards-based information exchange. ONC could then use those findings to establish a more quantitative survey instrument or other measurement tool to further measure usability and the impact of standards on usability. The findings from these qualitative and quantitative measures could inform the work of the Interoperability Standard Advisory Taskforce. They also should be used to help direct the work of ONC on driving usable interoperability.

6) Would health IT developers, exchange networks, or other organizations who are data holders be able to monitor the implementation and use of measures outlined in the report? If not, what challenges might they face in developing and reporting on these measures?
If such monitoring cannot be done, we have concern that like interoperability itself the burden will be passed down to the physician and their staff. ONC should be extremely cognizant of the potential administrative burdens placed on physicians if they are the entities that must monitor and measure interoperability.

It is high time vendors are required to report on something, be held accountable for accuracy of reporting, and be held accountable for creation of highly usable products that support natural clinician workflows Ð or else face the associated repercussions that may now unfold tied to a product which was not developed with user-centered design principles that results in low usability of functionality that has been developed with bare bones efforts strictly to meet certification requirements. We caution, however, that should reporting be implemented in a mandatory manner rather than via a voluntary or incentivized voluntary basis, ONC should be cognizant of the fact that onerous reporting requirements that exceedingly increase costs to HIT vendors and health information exchange organizations will result in increased costs being passed on to physicians.

For measure category a) standard used by end user, this absolutely should be monitored and reported by health IT vendors (not clinicians). There should be a means of ensuring integrity and accuracy within this reporting Ð in a similar way to how ONC has required QCDRs to have data validation plans to monitor and ensure accuracy of data, upon which ONC bases assessments which impact incentives earned within the EHR incentive program. We would strongly recommend that ONC consider conflicts of interest of entities monitoring and measuring their own interoperability. Therefore, ONC should consider a process to verify the measurement, such as random, routine audits of developers and networks.

7) Ideally, the implementation and use of interoperability standards could be reported on an annual basis in order to inform the Interoperability Standards Advisory (ISA), which publishes a reference edition annually. Is reporting on the implementation and/or use of interoperability standards on an annual basis feasible? If not, what potential challenges exist to reporting annually? What would be a more viable frequency of measurement given these considerations?
We would believe that an annual reporting would be sufficient and feasible.

8) Given that it will likely not be possible to apply the measurement framework to all available standards, what processes should be put in place to determine the standards that should be monitored?
We would recommend using the standards that support the key data that has shown to impact the quadruple AIM either in a negative way when not available or in a positive way when available. If such empirical evidence were insufficient to complete the prioritization, then we would recommend that a group of clinicians be convened to complete the prioritization process.
10) What measures should be used to track the level of ÒconformanceÓ with or customization of standards after implementation in the field?
We have some serious concern about the use of the phrase Òcustomization of standards after implementation.Ó Customization of a standard should only be done by its standards development organization (SDO). As for constraining optionality within a standard, the industry has used the process of creating implementation guides, which can take a standard designed to support numerous use cases and provide guidance on how the optionality in the standard should be used to support a specific (or few) use case(s). We would encourage ONC to continue to take advantage of the implementation guide creation process. As for conformance, measures should focus both on conformance to the standard and to specific implementation guides for the standard.

Having a highly technical communication tool is immeasurably valuable- and useful in thousands of existing benchmark quantitative statements and scenarios. However the brutal reality of transferring the costs of initiating and maintaining EHR, subsequent interoperability and then having an inaccessibility experience with personal medical record retrieval is shocking! The technology implementation cost has just contributed to enormous ##ACA $$, is now considered basic essential as working class with > 75,000 on duty dedicated hours employment seeking basic quality care, the costs of care are beyond daunting even insurmountable!