WellSky® Submits Comment Letter to U.S. Department of Health and Human Services Regarding Health Data, Technology, and Interoperability Proposed Rule
June 20, 2023
Micky Tripathi, Ph.D., M.P.P.
National Coordinator for Health Information Technology
Office of the National Coordinator for Health Information Technology
U.S. Department of Health and Human Services
Attention: RIN 0955–AA03
Mary E. Switzer Building
Mail Stop: 7033A
330 C Street SW
Washington, DC 20201
Re: Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing Proposed Rule (RIN 0955-AA03)
Dear Dr. Tripathi:
WellSky appreciates the opportunity to offer our comments in response to the Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing Proposed Rule (the “Proposed Rule”).[1] Founded over 40 years ago, WellSky is a healthcare solutions company that serves one in three home-based care providers, and powers a network of 130,000+ home- and community-based providers employing 1,000,000+ nurses, physical and occupational therapists, social workers, home health aides, and caregivers. WellSky’s national network paired with real-time analytics enables a care coordination platform that can deploy both skilled and non-skilled home-based care interventions to individuals across the country.
As a provider of innovative technology, WellSky is a strong proponent of interoperability and the power of appropriate predictive analytics and decision support technology and its role in optimizing quality of care in an environment of limited clinical capacity. We align with and applaud the intent to ensure that any models deployed are fair, appropriate, valid, effective, and safe (FAVES). We also care deeply about making technology that is as usable and unintrusive at the point of care as possible, so that it increases the efficiency of care delivery and enhances the quality of care while minimizing administrative friction for participants. As such, we are compelled to provide constructive feedback to aid in ensuring that the final rule can address its objectives and avoid unintended consequences that stifle innovation and reduce adoption of the very technologies we collectively expect will support transformation in outcomes and efficiency in health care.
As there are many components of the Proposed Rule, WellSky will focus on areas in which we have the greatest concern. WellSky appreciates the summary’s acknowledgement of the desire to ensure high quality models can be identified in the market by providers and other potential users. We share that desire as a developer of high quality and well-balanced models. As initially proposed, WellSky posits that some of the proposed transparency requirements may in practice work against this goal, which is detailed in the comments below regarding intellectual property and performance.
WellSky also supports the summary’s observation that the rule intentionally does not propose baselines, measures or thresholds at this time, but rather focuses on access to appropriate information to inform decision-making. We expand on these general comments in the sections below.
§ 170.315(a)(9): Extending the Deadline for Clinical Decision Support Interventions
Throughout the Proposed Rule, the timeline for certifying to the new definition of the Base EHR is proposed as “by December 31, 2024.” Given the scope of work proposed related to developing accessible source attribute information inclusive of intervention details, development, performance, and maintenance along with risk management information, and given that the final rule will not be out until late in calendar year 2023, and considering that the real-world testing plans for prior certified developers are due by December 15, 2023, WellSky strongly notes that fewer than six months to plan and fewer than 18 months to execute whatever is finalized in the new rule is insufficient time. We suggest finalizing a timeline for “by December 31, 2025” to address appropriate implementation timelines.
§ 170.315(b)(11): Decision Support Interventions (DSI)
Feedback loops (b)(11)(ii)(C)
The summary expresses a desire to encourage feedback loops from users of DSIs. WellSky supports and already enables users to provide feedback on its solutions via in-app tools. Further, WellSky monitors user analytics in our applications and cross-references usage with quality metrics to assess DSI impact on patient outcomes. However, the Proposed Rule posits that users, as opposed to developers, will engage in evaluating and improving DSI performance, and as such proposes requiring data regarding “action taken” for every DSI instance. In practice, requiring users to document an action taken for every DSI instance would result in burdensome workflows and extra clicks, rendering solutions more cumbersome for users and hindering adoption and impact of otherwise transformative tools and features. WellSky does not support requiring a documented action for every DSI in a solution.
In addition, the Proposed Rule is unclear regarding the scope and duration of data that could be downloaded and by whom. WellSky suggests that no more than two years of such data be required. A user from one provider organization should not be permitted to download feedback data from a different organization. Further, feedback data should not be available to the public and thus the developer’s competitors.
Source attributes (b)(11)(vi)
The ONC requested comment on whether source attributes need to be publicly available and/or machine readable. WellSky does not support requiring public availability, as developers will already be naturally incentivized to share their information with prospects in addition to clients. Making this information fully public would exacerbate intellectual property (IP) and competitive concerns discussed herein. Given the intentional omission of proposed defined baselines, measures, or thresholds, which WellSky supports, we believe that it is premature to define standardized formats and machine-readable files. Requiring machine readable files of non-standardized source attributes would create a large administrative and technical burden for developers while providing little value or insight to potential users. WellSky does not support public posting or machine-readable files for source attribute information.
Section § 170.315(b)(11)(vi) proposes that applicable DSIs must enable the user to review source attribute information via “direct display, drill down, or link out…” WellSky supports all three modes and encourages the ONC to maintain the “drill down” and “link out” options in addition to “direct display.” WellSky has found that including too much information in the direct display can negatively impact usability and user adoption in comparison to providing rational and accessible paths to deeper information via click-paths that are based on user-centered design principles. Further, WellSky interprets this provision to mean that the user would see the specific values of the primary contributing source attributes for this particular prediction, not a list of all source attributes that are predictive for the model — of which there could be hundreds. We comment in the following Section§ 170.315(b)(11)(vi)(C) regarding concerns with intellectual property and sharing proprietary model attributes in their entirety.
170.315(b)(11)(vi)(C)
By cross-reference, Section § 170.315(b)(11)(vi)(C) proposes that Health IT Modules enabling or interfacing with predictive DSIs also make accessible the entirety of the source attributes in paragraph (b)(11)(vi)(A), which describes the requirements for evidence-based DSIs. Given that (b)(11)(vi)(A)(1)-(4) are specific to evidence-based DSIs and only (5)-(7) would be applicable to predictive DSIs, we propose removing the reference to (b)(11)(vi)(A) from (b)(11)(vi)(C).
Specific to (b)(11)(vi)(C)(2) and (3), WellSky supports transparency in sharing information regarding intervention development and performance, and in fact, already provides white papers for applicable predictive models in our solutions. However, publicly disclosing the exact lists of data elements (versus data classes) in training data sets, along with identifying which elements turned out to be predictive when paired with quantitative measures of intervention performance, essentially removes IP protections, and in essence, motivations for innovation in healthcare decision support development. WellSky encourages the ONC to clarify that listing proprietary itemized data elements is not required. Disclosing data classes that are included in the model (for example “demographics” or “social determinants of health”) rather than itemized elements would suffice for transparency while still protecting IP. The Proposed Rule commentary indicates that developers would also be required to expose how cut-points were selected. The IP in evaluating and determining how to stratify model output is significant and as such this should not be finalized in the rule.
WellSky supports the commentary on § 170.315(b)(11)(vi)(C)(3)(i) “Validity of prediction in test data,” which indicates that the proposal does not prescribe specific performance or validation measures. To keep the desired flexibility but ensure chosen measures are not based on cherry-picked subsets, WellSky suggests that the ONC add language that ensures the chosen measures reflect population-wide performance measurement in the intended populations. This will preclude inappropriate reference population narrowing to find metrics that support positive performance metrics, which would make performance comparison difficult for prospective users.
Regarding § 170.315(b)(11)(vi)(C)(3)(ii) “Fairness of prediction in test data,” WellSky supports the need to aid users in understanding whether models have been tested for fairness or bias. To accomplish this while protecting IP we suggest that the ONC consider requiring that developers publish anti-bias testing results on specific dimensions of concern (for example race, ethnicity, sex) and explicitly not require the developer to disclose whether any given element is a material predictive input.
WellSky supports the proposed use of “if available” in (b)(11)(vi)(C)(3) and (4). Given that external and/or local data are not always available in all circumstances, this qualifier is needed. WellSky invites a more specific definition of “local” since one provider organization may have broad geographic location coverage.
170.315(b)(11)(vi)(D)
WellSky does not agree that a developer of certified health IT that interfaces with a third party predictive DSI has any accountability for that third-party model, which in many cases may have been selected, purchased, and/or developed independently by the provider organization rather than the developer. It should be the responsibility of the third party to display and describe source attributes and other information as required. The developer of the other system, at most, could denote if a DSI it interfaces with is in fact a third-party model, thus informing the user of the need to seek out any desired information from the primary developer of the DSI in question. Communicating that a model is third-party is sufficient, while the proposed language of saying source attribute information is “not available for user review” is both unnecessarily pejorative to the third-party and misleading to the end user.
170.315(b)(11)(vi)(E)
The proposal to “Enable a limited set of identified users to author and revise source attributes and information beyond source attributes listed in paragraphs (b)(11)(vi)(A) and (b)(11)(vi)(C) of this section, as applicable” is noted in the rule commentary as intended to support ongoing evolution of source attributes. This fails to define the intent for or characteristics of the limited set of identified users and would enable those users to create extra regulatory requirements for developers of certified health IT. Any such requirements would not be evenly applied to all such developers since different users would provide different feedback to different developers, which would then be difficult to aggregate and synthesize into updated universal requirements. WellSky strongly discourages this approach to user input on regulatory evolution.
Section § 170.315(b)(11)(vii)
In keeping with our comments under Section § 170.315(b)(11)(vi)(D) oversight of third-party models, WellSky supports intervention risk management practices for developers of certified health IT. However, the proposed language is overly broad by the inclusion of “or interface with.” The ONC requested comment on whether one developer of certified health IT can or should have oversight over third-party solutions. In the scenario where a certified Base EHR is exchanging data with a certified or not-certified third-party DSI, it is neither feasible nor competitively rational for one developer to perform risk analysis, mitigation or governance over the decision support tools developed by a third party. Purchasers/users of third-party tools should expect such information directly from the developer, and the final rule should clarify that a given developer of certified health IT is accountable for risk management practices for its own DSI solutions but cannot be accountable for third parties.
In (b)(11)(vii)(C), summary risk management practices are proposed to be publicly available. WellSky notes that analogous security evaluations like SOC 2 are not required to be made public. Given the other elements of the Proposed Rule and the demand from prospects and clients to review and assess, non-public risk management information from the developer will be sufficient to ensure that developers have adequate risk management practices. WellSky does not see additive value in making such information public and instead anticipates that such a requirement would result in a high administrative burden and risk to proprietary information while producing little public value.
§ 170.102: Narrow the Definitions Related to Decision Support Interventions and Predictive Models
The ONC requested feedback on whether the proposed definitions effectively delineate an understanding of “predictive decision support interventions.” The proposed definition states, “Predictive decision support intervention means technology intended to support decision-making based on algorithms or models that derive relationships from training or example data and then are used to produce an output or outputs related to, but not limited to, prediction, classification, recommendation, evaluation, or analysis,” while further explanation in the summary references examples like lists or forms and notes a broad interpretation of “intended to support decision-making.” As currently described, the proposed definition could be interpreted to classify any list of patients, information form, or even a comparison against a population average as predictive DSI. The final rule should remove the overly broad examples and/or clarify that the definition applies only when the predictive modifier applies (“based on algorithms or models that derive relationships…”).
§ 170.213: Updating the United States Core Data for Interoperability Standard (USCDI) v3
The ONC proposes to update the USCDI standard in Section § 170.213 by adding the newly released USCDI v3 and by establishing a January 1, 2025, expiration date for USCDI v1 (July 2020 Errata) for purposes of the Program. The ONC proposes to add USCDI v3 in § 170.213(b) and incorporate it by reference in § 170.299. Specifically, USCDI v3 in this Proposed Rule refers to the USCDI v3 (October 2022 Errata). The ONC proposes to codify the existing reference to USCDI v1 (July 2020 Errata) in § 170.213(a). The ONC proposes that as of January 1, 2025, any Health IT Modules seeking certification for criteria referencing § 170.213 would need to be capable of exchanging the data classes and data elements that comprise USCDI v3.
WellSky supports the thesis that the new elements required in the USCDI v3 are critical for planning patient-centered care and should be exchanged between providers during transitions of care to support proper care planning and ensure optional outcomes for patients. However, Health Status Assessments are often created and customized by providers and therefore vary from provider to provider. While obtaining these assessments from the standardized Medicare assessments (OASIS, IRF-PAI, MDS, etc.) should be achievable by January 1, 2025, it is not feasible to require these data elements from custom, often non-standardized, non-codified forms that likely vary from provider to provider and organization to organization by January 1, 2025.
§ 170.315(d)(14): Remove Burdens Associated with the Patient Requested Restrictions Certification Criterion
ONC seeks public comment on each part of this proposal—the new criterion in § 170.315(d)(14), the inclusion of the request capability for patients in § 170.315(e)(1), and the requirements with the Privacy and Security Framework in § 170.550(h)—both separately and as a whole. ONC specifically seeks comment on the feasibility of each part in terms of technical implementation and usefulness for patients and covered entities using these capabilities. The ONC also seeks comment on the health IT development burden associated with implementation of the capabilities including for the individual certification criterion referenced in the Privacy and Security Framework in § 170.550(h).
While we applaud the ONC’s desire to protect health data privacy while also striving to improve the interoperability and sharing of data, the technology and policy challenges of executing a discrete level of data release management are extremely complex. If the developer has the flexibility to implement the restriction on the inclusion in a subsequent use or disclosure via a wide range of potential means, dependent on their specific development and implementation constraints (e.g. flagged would not be included as part of a summary care record, not to be displayed in a patient portal or not to be shared via an API), patients will be confused from potential provider to provider on what they are actually protected from under privacy rules versus what is the policy of a particular technology in which their data resides. The breach of protected health information (PHI) is treated, appropriately, with a high degree of scrutiny and can carry significant penalties and implications to the reputation of an organization. Consideration should be given to protection against accidental disclosure (a medication disclosed through notes, a patient’s condition indicated by their child’s family history) to prevent discouraging interoperability for fears of liability. Furthermore, consideration must be given to the ability to override requests in the case of a patient’s safety.
The proposed addition of “patient requested restrictions” to § 170.315(d)(14) does put an additional burden on the health IT developer to ensure that its health IT solution can manage and enforce the proposed restrictions in an effective, automatic and usable manner. While existing restrictions are currently enforced at the patient record level, a fine-grained approach will require the re-architecture of most core systems of health IT solutions. This can lead to significant additional costs. According to a study from the University of Southern California that evaluated the impact of analogous efforts, that cost can be anywhere from 20% – 80% added effort on a project[2]. Additionally, the development of consent management and enforcement components is currently not generally available or does not show the level of maturity necessary to meet the proposed requirements. While agreeing with the direction, we propose that the health IT developers need additional time to implement these additions. The proposed deadline of January 1, 2026, to enforce compliance with these new restrictions within a health IT solution will seriously limit the number of innovations that developers can bring to market in addition to those changes. Touching every internal aspect of the health IT solution to ensure that restrictions are enforced is orders of magnitude more complex than driving interoperability standards at its periphery.
In the Alignment with Adopted Standards, the ONC is looking for comments on whether the restrictions proposed in § 170.315(d)(14) should provide more implementation guidance over flexibility. The ONC seeks feedback on the following three alternatives:
- In section III.C.10.b.i, we seek comment on a set of alternate proposals adopting each of the HL7 DS4P IGs, the HCS Security Label Vocabulary, or both for the new criterion in § 170.315(d)(14).
- In section III.C.10.b.ii, we seek comment on alternate proposals adopting the HL7 DS4P IGs and/or the HCS Security Label Vocabularies with constraints beyond those described in the IGs, that, if finalized, would constrain the requirements within the IGs to only certain use cases.
- In section III.C.10.b.iii, we seek comment on an additional alternate proposal that, if finalized, would limit the specified scope of USCDI data that the proposed new criterion in § 170.315(d)(14) and the proposed revised criterion in § 170.315(e)(1) would be required to support.
It is helpful for health IT developers to rely on existing standardized concepts given that many health IT developers will have to retrofit existing systems with those capabilities. Furthermore, many health IT developers will have to rely on offerings provided by third-party partners that will benefit from a standardized approach. We would suggest an incremental approach to applying the standards to first a specified scope of USCDI data that can then be increased over time, similarly to how the data unblocking mandate was structured. This approach would allow health IT developers to learn how practically those security labels work at scale in their systems. Allowing a phase of review and inspection before extending to a broader scope would assist in minimizing risk.
It is not clear whether limiting the application of the requirements to certain use cases would dramatically reduce the burden on the health IT developers. The bulk of the burden is to govern the access and use of the data element, the scenario under which this access or use occurs does add extra complexity from a usability point of view, as now users and patients must understand under which circumstances a data element might be restricted and under which it is not.
We also want to bring to the attention of the ONC that “bolting on” new security capabilities on top of systems that have not been designed for this can have unintended consequences from a usability, performance and complexity standpoint. One such issue is how easy it is for one to understand the effective permission that someone has over a data element. Fine-grained access control, combined with existing coarse-grained policies, is likely to be complicated to understand for both health IT solutions users and patients. These are similar to issues that other types of IT systems have experienced for years.
Conclusion
WellSky appreciates the opportunity to comment on the Proposed Rule. We urge the ONC to work with the industry as the agency finalizes this rule to avoid the unintended consequences of the proposed transparency and interoperability provisions while achieving the aims of the Office. Additionally, we encourage continued support of ongoing innovation in predictive analytics that drive transformation in healthcare quality and efficiency.
[1] 88 Fed. Reg. 23746 (April 18, 2023).
[2] University of Southern California: Cost of Developing Secure Software