Koios DS (3.8)

{0} FDA U.S. FOOD & DRUG ADMINISTRATION July 2, 2026 Koios Medical, Inc. Michael Bocchinfuso Director of Regulatory Compliance and Quality 242 W. 38th St. 14th Floor New York, New York 10018 Re: K253884 Trade/Device Name: Koios DS (3.8) Regulation Number: 21 CFR 892.2060 Regulation Name: Radiological Computer-Assisted Diagnostic Software For Lesions Suspicious Of Cancer Regulatory Class: Class II Product Code: POK, QIH Dated: June 4, 2026 Received: June 4, 2026 Dear Michael Bocchinfuso: We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading. If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register. U.S. Food & Drug Administration 10903 New Hampshire Avenue Silver Spring, MD 20993 www.fda.gov {1} K253884 - Michael Bocchinfuso Page 2 FDA's substantial equivalence determination also included the review and clearance of your Predetermined Change Control Plan (PCCP). Under section 515C(b)(1) of the Act, a new premarket notification is not required for a change to a device cleared under section 510(k) of the Act, if such change is consistent with an established PCCP granted pursuant to section 515C(b)(2) of the Act. Under 21 CFR 807.81(a)(3), a new premarket notification is required if there is a major change or modification in the intended use of a device, or if there is a change or modification in a device that could significantly affect the safety or effectiveness of the device, e.g., a significant change or modification in design, material, chemical composition, energy source, or manufacturing process. Accordingly, if deviations from the established PCCP result in a major change or modification in the intended use of the device, or result in a change or modification in the device that could significantly affect the safety or effectiveness of the device, then a new premarket notification would be required consistent with section 515C(b)(1) of the Act and 21 CFR 807.81(a)(3). Failure to submit such a premarket submission would constitute adulteration and misbranding under sections 501(f)(1)(B) and 502(o) of the Act, respectively. Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download). Your device is also subject to, among other requirements, the Quality Management System Regulation (QMSR) (21 CFR Part 820), which includes, but is not limited to, ISO 13485 clause 7.3 (Design controls), ISO 13485 clause 8.3 (Nonconforming product), ISO 13485 clause 8.5.2 (Corrective action), and ISO 13485 clause 8.5.3 (Preventative action). Please note that regardless of whether a change requires premarket review, the QMSR requires device manufacturers to review and approve changes to device design and production (ISO 13485 clause 7.3 and ISO 13485 clause 7.5) and document changes and approvals in the Medical Device File (ISO 13485 clause 4.2.3). Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the Quality Management System Regulation (QMSR) (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050. All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device {2} K253884 - Michael Bocchinfuso Page 3 Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/unique-device-identification-system-udi-system. Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medical-device-reporting-mdr-how-report-medical-device-problems. For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100). Sincerely, Jessica Lamb, Ph.D. Assistant Director DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health Enclosure {3} | Indications for Use | | | | --- | --- | --- | | Please type in the marketing application/submission number, if it is known. This textbox will be left blank for original applications/submissions. | K253884 | ? | | Please provide the device trade name(s). | | ? | | Koios DS (3.8) | | | | Please provide your Indications for Use below. | | ? | | Koios Decision Support (DS) is an artificial intelligence (AI)/machine learning (ML)-based computer-aided diagnosis (CADx) software device intended for use as an adjunct to diagnostic ultrasound examinations of lesions suspicious for breast or thyroid cancer.Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an AI/ML-derived cancer risk assessment and selects applicable lexicon-based descriptors designed to improve overall diagnostic accuracy as well as reduce interpreting physician variability.Koios DS software may also be used as an image viewer of multi-modality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and document images, and output into a structured report.Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. When utilized by an interpreting physician who has completed the prescribed training, this device provides information that may be useful in recommending appropriate clinical management.Limitations:Patient management decisions should not be made solely on the results of the Koios DS analysis.Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them.Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images.The software does not predict the presence of the thyroid nodule margin descriptor, extra-thyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list. | | | | Please select the types of uses (select one or both, as applicable). | Prescription Use (21 CFR 801 Subpart D)Over-The-Counter Use (21 CFR 801 Subpart C) | ? | {4} K253884 # 510(k) Summary This 510(k) summary of substantial equivalence information is submitted as part of the Premarket Notification in accordance with the requirements of 21 CFR Part 807, Subpart E and Section 807.92. # 1. Identification of Submitter: | Submitter: | Koios Medical, Inc. | | --- | --- | | Address: | 242 West 38th Street, 14th Floor New York, NY 10018 | | Phone: | 732-529-5755 | | Fax: | 732-529-5757 | | Contact: | Michael Bocchinfuso | | Title: | Director of Regulatory Compliance and Quality | | Phone: | 732-529-5755 | | Fax: | 732-529-5757 | | Summary Date: | November 5, 2025 | # 2. Identification of Product: | Trade Name: | Koios DS (3.8) | | --- | --- | | Device Common Name: | Radiological Computer-Assisted Diagnostic Software | | Device Classification: | 21 CFR 892.2060, Class II, POK (primary) 21 CFR 892.2050, Class II, QIH (secondary) | | Classification Name: | Radiological Computer-Assisted Diagnostic Software (CADx) for Lesions Suspicious for Cancer | | Manufacturer: | Koios Medical, Inc. | # 3. Marketed Devices In terms of safety and performance, this software medical device is substantially equivalent to the devices listed below: {5} 2 | Predicate device: | Koios DS | | --- | --- | | Manufacturer: | Koios Medical, Inc. | | 510(k) Number: | K242130 | ### 4. Device Description Koios Decision Support (DS) is a software application designed to assist trained interpreting physicians in analyzing breast and thyroid ultrasound images. The software device is a web application that is deployed to a Microsoft IIS web server and accessed by a user through a compatible client. Once logged in and granted access to the Koios DS application, the user examines selected breast or thyroid ultrasound DICOM images. The user selects Regions of Interest (ROIs) of orthogonal views of a breast lesion or thyroid nodule for processing by Koios DS. The ROI(s) are transmitted electronically to the Koios DS server for image processing and the results are returned to the user for review. #### Breast Functionality: Koios DS software automatically classifies breast lesions suspicious for cancer based on image data into one of four ACR BI-RADS® Atlas[1] or European U1-U5 Classification System-aligned categories (Benign, Probably Benign, Suspicious or Indeterminate, or Probably Malignant) and also displays a continuous graphical Confidence Level Indicator depicting where the lesion falls within its respective category and its relation to neighboring categories. The software automatically classifies the shape (Round, Oval, Irregular) and orientation (Parallel, Not Parallel) of the selected lesion. #### Thyroid Functionality: Koios DS is a software medical device used to analyze ultrasound data to classify user-selected regions containing thyroid nodules suspicious for cancer. The software generates a set of user-editable sonographic nodule descriptor recommendations (Composition, Echogenicity, Shape, Margin, Echogenic Foci) along with an optional, deep-learning derived cancer risk assessment of the suspected nodule from two orthogonal views. Nodule descriptor recommendations are subsequently mapped to a categorical assessment and risk level rating via the ACR TI-RADS™ ATLAS or American Thyroid Association (ATA) risk stratification systems (RSSs) based on user preference. The software's direct, non-descriptor-based cancer risk assessment is presented as the Koios "AI Adapter" that, when used in conjunction with the ACR TI-RADS or ATA guidelines for nodule risk stratification, is shown to improve overall diagnostic performance of both systems. The AI Adapter operates as an optional lexicon-specific input used to modify the final categorization in the ACR TI-RADS and ATA RSSs. The AI adapter positively impacts performance through either a point-based modification (either positive or negative) or a risk-shift modification (either positive or negative) for ACR TI-RADS and the ATA systems, respectively. This process creates an AI-augmented categorization that is meant to be used with no other modifications to the {6} 3 decision-making pathway of either RSS. A trained interpreting physician may choose to incorporate or exclude the Koios AI Adapter from the overall assessment when finalizing their diagnostic interpretation. [1] BI-RADS® ATLAS is a registered trademark of American College of Radiology. All Rights Reserved. Koios DS enables the following functionality: - Breast and Thyroid Diagnostic Core AI Engines enabled by state-of-the-art computer vision and machine learning techniques capable of reading, interpreting, analyzing, classifying and generating findings from ultrasound image data resulting in an automated risk assessment for breast lesions and thyroid nodules suspicious for cancer. - Automatic classification of thyroid nodule TI-RADS and ATA Descriptors of: Composition, Echogenicity, Shape, Margin, and Echogenic Foci based on user-selected regions of interest (ROIs). - Automatic classification of breast lesion BI-RADS and U1-U5 Descriptors Shape and Orientation based on user-selected or confirmed regions of interest (ROIs). - Annotation and description of ultrasound images based on ACR BI-RADS Breast Imaging Atlas and U1-U5 for Koios DS Breast and ACR TI-RADS Atlas for thyroid lexicon classification forms and ATA classification guidelines for Koios DS Thyroid. - Reporting forms for breast lesion or thyroid nodule identification and tracking in the Electronic Health Record. - Smart Calipers - extraction of user-supplied ROI data (alternately referred to as Calipers) embedded in DICOM SR files from the ultrasound modality. - Smart Click - for streamlining the manual ROI selection process. The Smart Click functionality enables the user to click on the center of a lesion in order to activate a system-generated region of interest surrounding the selected lesion for the user. - Image Registration and Matching - allows users to select images and regions of interest through their own image viewers when interacting with Koios DS Breast and Koios DS Thyroid, and facilitates a flexible viewer agnostic workflow. When the Image Matching {7} 4 Engine is given a screenshot of a medical image with coordinates for a region of interest, it identifies the original full quality image and translates the coordinates to its frame of reference. - Automatic Size and Position population using Optical Character Recognition (OCR) - the Koios DS Optical Character Recognition engine uses machine learning and rule-based methods to create a system which is capable of retrieving fast, accurate transcriptions of the text overlaid on ultrasound images. Given an ultrasound image that has been annotated by a radiologist or technician, the OCR function identifies all text in the image and extracts relevant information to the documentation of lesions or nodules. This allows users to quickly interpret and transcribe the locations and measurements of ultrasound findings. - Remote analysis interface to generate and view results within compatible software (e.g. ultrasound equipment or PACS workstation software). - Installer and Configuration Wizard. - Single Sign-on (SSO) Windows and LDAP Authentication. - Operating system and platform-agnostic usage. - Zero-footprint web-based HTML5 DICOM image viewer with image manipulation and annotation tools. - Ability to save findings to PACS. - Ability to export findings to reporting software. **User Profile:** Koios DS is for use by trained professionals only. Koios DS is not for use by patients. Users must have appropriate medical professional competence, such as trained sonographers and interpreting physicians. **Use Environment:** Koios DS is a software application for use within the healthcare setting (in a clinic or hospital) for the examination and assessment of breast lesions or thyroid nodules using ultrasound. It is a platform-agnostic web application that queries and accepts DICOM compliant digital medical files from any compliant device subject to the specified DICOM Conformance Statement for Koios DS. Processing of the image(s) occurs in conjunction with a trained interpreting physician's typical diagnostic case read. The output of the system is a digital display to be used as a {8} 5 concurrent read and report input that may be added as an addendum to the DICOM series selected for processing or exported directly into a patient's draft report. ### Operating Principle: Koios DS is an ASP.NET web application deployed to a web server inside a Windows operating system environment accessed by a user through a compatible client. The application provides image-derived data via web triggering and remote analysis. Once logged in and granted access to the Koios DS application, the user examines selected breast and thyroid ultrasound DICOM images. For breast functionality, the user selects or confirms up to two ROIs, from up to two orthogonal views that represent a single breast lesion for processing by the system. For thyroid functionality, two ROIs are required for processing by the system. The first ROI must be drawn on the transverse view, with the second on the longitudinal view of the nodule. For breast functionality, bench testing has verified a single ROI does not significantly decrease system AUC performance. The ROI(s) are transmitted electronically to the Koios DS server by the Koios DS Breast or Thyroid software for image processing and the results are returned to the user for review in the respective interface. Images and data can be stored, communicated, processed, and displayed within the system and/or across computer networks at distributed locations. The Koios DS Client is an optional workflow enhancement tool installed as a desktop application on the user workstation that enables a user to draw ROIs natively within their image viewing software. The Koios DS Client captures a screenshot of the ROI selected by the user instead of being directly drawn on and captured with DICOM data. The ROI screenshot is transmitted electronically to the Image Matching Engine within the Koios DS Server. The Image Matching Engine processes the ROI screenshot and data, identifying and matching the correct DICOM image, and overlaying the ROI on that image. Once matched, the ROIs are returned to the user for review in the Koios DS Breast or Thyroid interface. The software does not require any specialized hardware to return a diagnostic output, but the time to process ROIs will vary depending on the hardware specifications. Koios DS contains two distinct AI/ML engines to characterize breast lesions and thyroid nodules. Based on the structured data that exists within the DICOM header for a patient study, the Koios DS system calls the corresponding engine for analysis of the identified lesion or nodule. Each system uses computer vision and machine learning techniques embedded within an engine capable of reading, interpreting, analyzing, and generating findings from ultrasound data. The underlying Breast and Thyroid engines draw upon knowledge learned from a large database of known cases, tying image features to their eventual diagnosis, to form a predictive model. {9} 6 Koios DS results can be saved or transferred in three separate ways: in-transit transmission, saving to Picture Archiving and Communication System (PACS), and exporting results to third-party reporting software. In-transit transmission may be utilized when users wish to share analyses across viewing workstations. Results can be stored in in-transit memory for a preset period of time defined by a system administrator. After that preset period of time, all results are wiped from the local memory. Another method of saving is storing a report in the patient study on the PACS. After single or multiple lesion or nodule analyses have been performed and ultimately accepted by a trained interpreting physician, Koios DS can export a summary report to PACS as an addendum to the DICOM study that was selected for processing. This report serves as future reference and aid in the comparison of cases requiring follow up. This functionality is strictly reserved for approved users and must be configured by a site administrator. Koios DS also supports exporting results to third-party reporting software to facilitate the reporting process. Saving or exporting preferences can be configured by the system administrator and user. ## 5. Indications for Use Koios Decision Support (DS) is an artificial intelligence (AI)/machine learning (ML)-based computer-aided diagnosis (CADx) software device intended for use as an adjunct to diagnostic ultrasound examinations of lesions suspicious for breast or thyroid cancer. Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an AI/ML-derived cancer risk assessment and selects applicable lexicon-based descriptors designed to improve overall diagnostic accuracy as well as reduce interpreting physician variability. Koios DS software may also be used as an image viewer of multi-modality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and document images, and output into a structured report. Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult ( \( \geq \) 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult ( \( \geq \) 22 years) patients with thyroid nodules suspicious for cancer. When utilized by an interpreting physician who has completed the prescribed training, this device provides information that may be useful in recommending appropriate clinical management. ### Limitations: - Patient management decisions should not be made solely on the results of the Koios DS analysis. {10} 7 - Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them. - Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images. - The software does not predict the presence of the thyroid nodule margin descriptor, extra-thyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list. {11} 8 ## 6. Substantial Equivalence Chart | Product | Koios DS 3.6 (K242130) | Koios DS 3.8 (subject device) | | --- | --- | --- | | **Physical Characteristics** | Software Package Operates on off-the-shelf hardware | Software Package Operates on off-the-shelf hardware | | **Storage** | Storage not supported | Storage not supported | | **Image Input** | DICOM | DICOM | | **Characteristics** | Decision support device used to assist in the assessment and characterization of breast lesions and thyroid nodules using US image data. | Decision support device used to assist in the assessment and characterization of breast lesions and thyroid nodules using US image data. | {12} 9 | Intended Use/Indications for Use | Koios Decision Support (DS) is an artificial intelligence (AI)/machine learning (ML)-based computer-aided diagnosis (CADx) software device intended for use as an adjunct to diagnostic ultrasound examinations of lesions suspicious for breast or thyroid cancer. Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an AI/ML-derived cancer risk assessment and selects applicable lexicon-based descriptors designed to improve overall diagnostic accuracy as well as reduce interpreting physician variability. Koios DS software may also be used as an image viewer of multimodality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and | Koios Decision Support (DS) is an artificial intelligence (AI)/machine learning (ML)-based computer-aided diagnosis (CADx) software device intended for use as an adjunct to diagnostic ultrasound examinations of lesions suspicious for breast or thyroid cancer. Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an AI/ML-derived cancer risk assessment and selects applicable lexicon-based descriptors designed to improve overall diagnostic accuracy as well as reduce interpreting physician variability. Koios DS software may also be used as an image viewer of multimodality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and | | --- | --- | --- | {13} 10 document images, and output into a structured report. Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. When utilized by an interpreting physician who has completed the prescribed training, this device provides information that may be useful in recommending appropriate clinical management. document images, and output into a structured report. Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. When utilized by an interpreting physician who has completed the prescribed training, this device provides information that may be useful in recommending appropriate clinical management. {14} 11 | Target Population (subset of above for comparison purposes) | Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. | Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. | | --- | --- | --- | {15} 12 | **Limitations for Use** **(subset of above for comparison purposes)** | Limitations: • Patient management decisions should not be made solely on the results of the Koios DS analysis. • Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them. • Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images. • The software does not predict the presence of the thyroid nodule margin descriptor, extra-thyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list. | Limitations: • Patient management decisions should not be made solely on the results of the Koios DS analysis. • Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them. • Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images. • The software does not predict the presence of the thyroid nodule margin descriptor, extra-thyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list. | | --- | --- | --- | | **Modality Used for Analysis** | Breast Ultrasound Data Thyroid Ultrasound Data | Breast Ultrasound Data Thyroid Ultrasound Data | {16} 13 | Input | Medical images provided in a DICOM format | Medical images provided in a DICOM format | | --- | --- | --- | | ROI Requirements | Breast The software requires a user to select up to two ROIs, from up to two orthogonal views, that represent a single lesion to be selected and processed. Thyroid Two ROIs that represent a single lesion to be selected and processed are required for analysis. The first ROI is drawn on the transverse view of the nodule. The second is drawn on the longitudinal view. | Breast The software requires a user to select up to two ROIs, from up to two orthogonal views, that represent a single lesion to be selected and processed. Thyroid Two ROIs that represent a single lesion to be selected and processed are required for analysis. The first ROI is drawn on the transverse view of the nodule. The second is drawn on the longitudinal view. | | Output (Breast) | Koios defined categorical and continuous outputs (confidence level indicator) that align to BI-RADS, U1-U5, and auto-classified shape and orientation. | Koios defined categorical and continuous outputs (confidence level indicator) that align to BI-RADS, U1-U5, and auto-classified shape and orientation. | {17} 14 | Output (Thyroid) | Koios DS software automatically classifies thyroid nodules suspicious for cancer based on image data generating an output aligned to either the TI-RADS or ATA classification guidelines. The system automatically generates user-modifiable nodule descriptors (Composition, Echogenicity, Shape, Margin, Echogenic Foci) and a direct, image-derived cancer risk assessment that is translated into an optional lexicon-specific modifier. | Koios DS software automatically classifies thyroid nodules suspicious for cancer based on image data generating an output aligned to either the TI-RADS or ATA classification guidelines. The system automatically generates user-modifiable nodule descriptors (Composition, Echogenicity, Shape, Margin, Echogenic Foci) and a direct, image-derived cancer risk assessment that is translated into an optional lexicon-specific modifier. | | --- | --- | --- | | Comparative Clinical Performance Testing (Breast) | Metric: AUC Cases: 900 Readers: 15 | Metric: AUC Cases: 900 Readers: 15 | | Comparative Clinical Performance Testing (Thyroid) | Metric: AUC Cases: 650 Readers: 15 | Metric: AUC Cases: 650 Readers: 15 | {18} 15 ## 7. Description of Similarities and/or Differences ### Intended Use/Indications for Use (IFU) The Intended Use and Indications for Use of the subject device remain unchanged from the predicate device Koios DS (K242130). Over time, the company has gathered additional data to train and improve the performance of the Koios DS Thyroid engine. The intention of this submission is to seek approval for the latest engine based on the performance data outlined below. {19} 16 Technological Characteristics Comparison Table | | Koios DS 3.6 (K242130) | Koios DS 3.8 (K253884) | | --- | --- | --- | | Diagnostic Engine | Breast (v. 3.0.0) | Breast (v. 3.0.0) | | | Thyroid (v. 2.2.0) | Thyroid (v. 3.0.0) | {20} 17 | **Workflow Enhancements** | Breast Smart Click | Breast Smart Click | | --- | --- | --- | | | Breast Smart Calipers | Breast Smart Calipers | | | Thyroid Smart Click | Thyroid Smart Click | | | Thyroid Smart Calipers | Thyroid Smart Calipers | | | Image Registration and Matching | Image Registration and Matching | | | OCR Automatic Size and Position Population | OCR Automatic Size and Position Population Realignment of 'Cystic or Almost Completely Cystic' descriptor behavior to ACR TI-RADS guidance | {21} 18  Input {22} 19 Per the respective device descriptions of Koios DS 3.8 and Koios DS (K242130), the following technical characteristics are included in the Koios DS 3.8 (K253884) version: Optional Shape Descriptor Workflow: Based on customer feedback, the user has the option via a user setting to use the exact nodule length, width, and height measurements or an applied aspect ratio to the width measurement instead of the engine-generated descriptor to generate a 'Shape' descriptor value. The nodule measurement values are drawn from the OCR Automatic Size functionality or entered manually by the user. ## Output Thyroid & Breast Engine Performance: When comparing thyroid functionality, the subject Koios DS Thyroid Engine demonstrates a significant categorical output performance increase in AUC (2.6%), a significant increase in sensitivity (1.7%), and a significant increase (1.1%) in specificity. A full comparison of performance testing can be found in Section 6. The Koios DS Breast performance is unchanged from the predicate device. Optional Shape Descriptor Workflow: The change to the Koios DS Thyroid Shape descriptor selection allows user-configurable calculation from nodule measurements, in addition to the engine selection. This change aligns with customer feedback and ACR TI-RADS guidelines and American Thyroid Association (ATA) guidelines, which specify a selection of Taller-than-wide if the aspect ratio is greater than the user-configurable threshold. This change effectively allows the user to automatically override the engine output with the measurement calculation per the threshold, as they would be able to do so manually via the previous user interface. The existing engine-based selection remains available and is used if measurements are not available. Realignment of 'Cystic or Almost Completely Cystic' descriptor behavior to ACR TI-RADS guidance: This change was made to realign the 'Composition' descriptor output by the Koios Thyroid Engine to the latest American College of Radiology (ACR) TI-RADS guidance – specifically, the guidance on the point scoring when a nodule is 'Cystic or Almost Completely Cystic.' The ACR TI-RADS {23} 20 guidance indicates that predominantly cystic or spongiform nodules are inherently benign. If these features are present in a nodule, no further points from other descriptors will be added, and the nodule is automatically classified as a TR1.¹ The predicate device incorporated this guidance by automatically setting all other descriptor values to 0 when 'Spongiform' was system-generated or user-selected. However, the predicate device did not apply this behavior if the 'Cystic or Almost Completely Cystic' value was generated or user-selected. The subject device, Koios DS 3.8, realigns the system with the ACR TI-RADS guidance by applying the same functionality that currently exists for a 'Spongiform' classification to a 'Cystic or Almost Completely Cystic' classification. Performance testing outlined in the next section demonstrates equivalence and non-inferiority for each new technical characteristic. ### 8. Performance Testing – Standalone Testing #### Breast Engine The breast diagnostic engine in Koios DS v3.8 remains unchanged with respect to the breast diagnostic engine from the predicate device, Koios DS v3.6. For context, the bench testing that was performed for the predicate device, Koios DS 3.6 breast engine, to ascertain the degree of concordance with trained interpreting physicians is outlined below. Ground truth for malignancy risk classification was determined by pathology or 1-year follow-up for cases that were not biopsied. The system was analyzed on 900 lesions from 900 different patients set aside from the system's training data for the purpose of validating performance. Each lesion was represented by two orthogonal images (e.g. radial and anti-radial), providing a total of 1800 images. An expanded validation set of 1014 cases, including these 900 and an additional 114 cases is used to test for dataset drift. System performance on the 900 cases reported an AUC of 94.5%, with a Sensitivity of 0.976 [0.960, 0.992] and a Specificity of 0.632 [0.588, 0.676]. Ground Truthing Process: Performance testing for the breast engine follows the primary endpoints of the breast pivotal study — specifically, diagnostic AUC against benign/malignant classification. Ground truth labels in this dataset use the same class definitions specified in the Koios ¹ Tessler F, Middleton W, Grant E et al. ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. Journal of the American College of Radiology. 2017. {24} 21 DS for Breast clinical retrospective reader study. Every case is labeled benign based on biopsy or one-year follow-up (as reported by the physicians providing the data), or malignant based on biopsy. | Koios DS Engine (Breast) Test | Engine Version 3.0.0 (Previous) <Predicate Koios DS v3.6> | Engine Version 3.0.0 (Current) <Subject Koios DS v3.8> | | --- | --- | --- | | 1: Malignancy Risk Classifier AUC | 0.945 [0.932, 0.959] | 0.945 [0.932, 0.959] | | 2: Categorical Output Sensitivity Specificity | 0.976 [0.960, 0.992] 0.632 [0.588, 0.676] | 0.976 [0.960, 0.992] 0.632 [0.588, 0.676] | | 3: Sensitivity to Region of Interest | 0.012 | 0.012 | | 4. Sensitivity to Transducer Frequency | High frequency (>=15MHz), AUC = 0.948 [0.917, 0.978] Low frequency (<15MHz), AUC = 0.940 [0.925, 0.956] | High frequency (>=15MHz), AUC = 0.948 [0.917, 0.978] Low frequency (<15MHz), AUC = 0.940 [0.925, 0.956] | {25} 22 | | | | | --- | --- | --- | | 5. Single Image vs Orthogonal Image Pair | Single Image: 0.932 [+/- 0.003] | Single Image: 0.932 [+/- 0.003] | | 6. Assessment of Categorical Agreement – Shape *(prior results continue to apply)* | 0.738 [0.679, 0.797] | | | 7. Assessment of Categorical Agreement – Orientation *(prior results continue to apply)* | 0.744 [0.675, 0.813] | | | 8. Operating Point | PLR: System= 2.661 [2.338, 2.984] NLR: System= 0.039 [0.013, 0.064] PPV: System= 0.708 [0.672, 0.743] NPV: System= 0.966 [0.944, 0.988] | PLR: System= 2.661 [2.338, 2.984] NLR: System= 0.039 [0.013, 0.064] PPV: System= 0.708 [0.672, 0.743] NPV: System= 0.966 [0.944, 0.988] | {26} 23 | 9. Data Set Drift Analysis - Malignancy Risk Classifier AUC | ROCAUC = 0.949 [0.936, 0.962] | ROCAUC = 0.949 [0.936, 0.962] | | --- | --- | --- | | 10. Data Set Drift Analysis - Categorical Output | Sensitivity = 0.973 [0.958, 0.989] Specificity = 0.659 [0.619, 0.700] | Sensitivity = 0.973 [0.958, 0.989] Specificity = 0.659 [0.619, 0.700] | {27} 24 | 11. Subgroup Analysis: Age (AUC) | N/A | Sensitivity | | | | --- | --- | --- | --- | --- | | | | | System | Physician | | | | <40 | 0.940 [0.881, 0.999] | 0.938 [0.836, 1.000] | | | | 40-60 | 0.977 [0.953, 1.000] | 0.955 [0.885, 1.000] | | | | 60+ | 0.981 [0.963, 0.999] | 0.931 [0.835, 1.000] | | | | Specificity | | | | | | | System | Physician | | | | <40 | 0.773 [0.686, 0.860] | 0.418 [0.086, 0.750] | | | | 40-60 | 0.610 [0.549, 0.671] | 0.393 [0.139, 0.648] | | | | 60+ | 0.510 [0.416, 0.604] | 0.366 [0.108, 0.624] | {28} 25 | 12. Subgroup Analysis: Ethnicity (AUC) | N/A | | System | Physician | | --- | --- | --- | --- | --- | | | | White | 0.950 [0.934, 0.965] | 0.838 [0.787, 0.889] | | | | Black | 0.943 [0.898, 0.989] | 0.861 [0.795, 0.926] | | | | Asian | 0.928 [0.885, 0.971] | 0.784 [0.724, 0.845] | | | | Hispanic | 0.918 [0.850, 0.986] | 0.829 [0.749, 0.908] | | | | Other | 0.938 [0.851, 1.000] | 0.913 [0.816, 1.000] | {29} 26 | 13. Subgroup Analysis: Age (Operating Point) | N/A | Sensitivity | | | | --- | --- | --- | --- | --- | | | | | System | Physician | | | | <40 | 0.940 [0.881, 0.999] | 0.938 [0.836, 1.000] | | | | 40-60 | 0.977 [0.953, 1.000] | 0.955 [0.885, 1.000] | | | | 60+ | 0.981 [0.963, 0.999] | 0.931 [0.835, 1.000] | | | | Specificity | | | | | | | System | Physician | | | | <40 | 0.773 [0.686, 0.860] | 0.418 [0.086, 0.750] | | | | 40-60 | 0.610 [0.549, 0.671] | 0.393 [0.139, 0.648] | | | | 60+ | 0.510 [0.416, 0.604] | 0.366 [0.108, 0.624] | {30} 27 | 14. Subgroup Analysis: Ethnicity (Operating Point) | N/A | Sensitivity | | | | --- | --- | --- | --- | --- | | | | | System | Physician | | | | White | 0.980 [0.964, 0.997] | 0.945 [0.879, 1.000] | | | | Black | 1.000 [1.000, 1.000] | 0.978 [0.890, 1.000] | | | | Asian | 0.945 [0.882, 1.000] | 0.916 [0.771, 1.000] | | | | Hispanic | 0.943 [0.866, 1.000] | 0.931 [0.763, 1.000] | | | | Other | 1.000 [1.000, 1.000] | 0.978 [0.777, 1.000] | | | | Specificity | | | | | | | System | Physician | | | | White | 0.638 [0.577, 0.699] | 0.422 [0.174, 0.671] | {31} 28 | | | Black | 0.647 [0.525, 0.769] | 0.332 [0.036, 0.627] | | --- | --- | --- | --- | --- | | | | Asian | 0.551 [0.441, 0.661] | 0.375 [0.153, 0.598] | | | | Hispanic | 0.632 [0.471, 0.792] | 0.402 [0.076, 0.728] | | | | Other | 0.750 [0.530, 0.970] | 0.443 [0.000, 0.917] | Bench testing demonstrates that the system exceeds physician performance measured by AUC, sensitivity, and specificity. The engine’s shape and orientation predictions have not been modified from the previously cleared device (which demonstrated the required level of agreement with the subjective categorizations assigned by physicians). Testing characterizes the system’s sensitivity to shifts in the selected region of interests (ROI) and transducer frequency. Testing characterizes the system’s Positive Predictive Value (PPV), Negative Predictive Value (NPV), Positive Likelihood Ratio (PLR) and Negative Likelihood Ratio (NLR) in comparison with physicians. Testing demonstrates that the performance of the engine does not demonstrate degradation when regions of interest are provided by the Smart Click or Smart Caliper system, as compared to manually drawn regions of interest. Therefore, the diagnosis engine is agnostic to the source of input ROI (Smart Click, Smart Calipers, physician drawn) and robust to shifts in ROI. In all tests, the Breast engine met or exceeded performance requirements. {32} 29 ## Thyroid Engine Bench testing was performed on the thyroid engine to ascertain the degree of concordance with trained interpreting physicians utilizing both the ACR TI-RADS and ATA classification systems. Ground truth for malignancy risk classification was determined by pathology results only. The system was analyzed on 650 nodules from 650 different patients set aside from the system's training data for the purpose of validating performance. Of the 650 cases, 500 were gathered from multiple test sites in the United States (US), while 150 were gathered from test sites in the European Union (EU). Each nodule was represented by two views (e.g. transverse and longitudinal), providing a total of 1,300 image views. Ground Truthing Process: Performance testing for the thyroid engine follows the primary endpoints of the thyroid pivotal study — specifically, diagnostic AUC against benign/malignant classification. Ground truth labels in this dataset use the class definitions specified in the Koios DS Thyroid clinical retrospective reader study. In summary, every case is labeled benign or malignant based on biopsy, fine needle aspiration, or surgical excision. When applied to diagnoses made using ACR TI-RADS guidelines, the AI Adapter and descriptor predictors achieved an AUC of 84.3%, demonstrating a significant increase over the average physician AUC. When recommending biopsy, the system's sensitivity is 0.700 [0.615, 0.785] and specificity is 0.629 [0.585, 0.672]. When recommending follow-up, the system's sensitivity and specificity are 0.892 [0.843, 0.942] and 0.533 [0.488, 0.578, respectively. In both scenarios, bench testing of the system demonstrates a non-significant improvement in sensitivity and a significant improvement in specificity over the physician average. Tests demonstrating AI Adapter impact on ATA classifications yielded similarly improved performance. With application of the AI Adapter, physician AUC demonstrates a significant increase of 8.731% [6.139, 11.324]. Sensitivity shows a non-significant increase of 1.345% [-3.419, 6.110], while specificity shows a significant increase of 28.493% [20.313, 36.673]. The below table provides a detailed evaluation of the thyroid engine across key performance metrics: {33} 30 | Test | Criteria | Engine version LIN.2.2.0 (Predicate device: Koios DS v3.6) | Engine version LIN.3.0.0 (Subject device: Koios DS v3.8) | | --- | --- | --- | --- | | 1 | Koios AI Adapter Standalone AUC | Non-Parametric AUC 0.869 [0.834, 0.904] Parametric AUC 0.872 [0.834, 0.910] | Non-Parametric AUC 0.869 [0.834, 0.904] Parametric AUC 0.872 [0.834, 0.910] | | 2 | AI Adapter Impact on AUC | Non-Parametric AUC 0.817 [0.773, 0.860] Parametric AUC 0.817 [0.770, 0.864] | Non-Parametric AUC 0.837 [0.796, 0.878] Parametric AUC 0.843 [0.799, 0.887] | | 3 | AI Adapter Impact on Operating Point (FNA) | Sensitivity 0.683 [0.605, 0.761] Specificity 0.618 [0.577, 0.659] | Sensitivity 0.700 [0.615, 0.785] Specificity 0.629 [0.585, 0.672] | | 4 | AI Adapter Impact on Operating Point (Follow-up) | Sensitivity 0.879 [0.812, 0.946] Specificity 0.495 [0.446, 0.544] | Sensitivity 0.892 [0.843, 0.942] Specificity 0.533 [0.488, 0.578] | {34} 31 | 5 | Sensitivity to Region of Interest |  |  | | --- | --- | --- | --- | | 6 | Adapter Sensitivity to Transducer Frequency | Low Frequency Data 0.776 [0.739, 0.813]High Frequency Data 0.892 [0.866, 0.918] | Low Frequency Data 0.813 [0.758, 0.868]High Frequency Data 0.885 [0.825, 0.945] | | 7 | Koios TI-RADS Prediction AUC | Non-Parametric AUC 0.746 [0.694, 0.798]Parametric AUC 0.746 [0.694, 0.799] | Non-Parametric AUC 0.787 [0.737, 0.836]Parametric AUC 0.787 [0.740, 0.834] | | 8 | Koios TI-RADS Prediction Operating Point (FNA) | Sensitivity 0.655 [0.573, 0.737]Specificity 0.430 [0.386, 0.474] | Sensitivity 0.685 [0.598, 0.771]Specificity 0.433 [0.389, 0.476] | | 9 | Koios TI-RADS Prediction Operating Point (Follow-Up) | Sensitivity 0.857 [0.788, 0.925]Specificity 0.250 [0.209, 0.291] | Sensitivity 0.892 [0.843, 0.942]Specificity 0.262 [0.224, 0.300] | {35} 32 | 10 | Koios TI-RADS Descriptor Agreement | Composition 0.502 [0.441, 0.562] Echogenicity 0.477 [0.433, 0.522] Margin 0.264 [0.213, 0.315] Shape 0.397 [0.306, 0.487] Comet-tail Foci 0.139 [0.078, 0.200] Macrocalcification 0.516 [0.439, 0.593] Peripheral Foci 0.448 [0.240, 0.656] PEF 0.322 [0.244, 0.399] | Composition 0.520 [0.426, 0.613] Echogenicity 0.520 [0.499, 0.591] Margin 0.309 [0.217, 0.400] Shape 0.463 [0.337, 0.589] Comet-tail Foci 0.139 [0.078, 0.200] Macrocalcification 0.516 [0.439, 0.593] Peripheral Foci 0.448 [0.240, 0.656] PEF 0.322 [0.244, 0.399] | | --- | --- | --- | --- | | 11 | Koios ATA Classification AUC | Difference in AUC 0.045 [0.028, 0.061] Percent Difference 6.137 [3.917, 8.356] Difference in Parametric AUC 0.064 [0.045, 0.084] | Difference in AUC 0.045 [0.028, 0.061] Percent Difference 6.137 [3.917, 8.356] Difference in Parametric AUC 0.064 [0.045, 0.084] | {36} 33 | | | Percent Difference 8.731 [6.139, 11.324] | Percent Difference 8.731 [6.139, 11.324] | | --- | --- | --- | --- | | 12 | Koios ATA Classification Operating Point (FNA) | Difference in Sensitivity 0.008 [-0.023, 0.039] Percent Difference 1.345 [-3.419, 6.110] Difference in Sensitivity 0.046 [0.033, 0.060] Percent Difference 28.493 [20.313, 36.673] | Difference in Sensitivity 0.008 [-0.023, 0.039] Percent Difference 1.345 [-3.419, 6.110] Difference in Sensitivity 0.046 [0.033, 0.060] Percent Difference 28.493 [20.313, 36.673] | | 13 | Independent Site Test – AI Adapter Standalone AUC | Non-Parametric AUC 0.858 [0.766, 0.950] Parametric AUC 0.856 [0.764, 0.947] | Non-Parametric AUC 0.877 [0.799, 0.955] Parametric AUC 0.898 [0.817, 0.980] | | 14 | Independent Site Test – AI Adapter + TI-RADS Prediction AUC | Non-Parametric AUC 0.843 [0.737, 0.949] Parametric AUC 0.862 [0.766, 0.958] | Non-Parametric AUC 0.874 [0.801, 0.946] Parametric AUC 0.887 [0.809, 0.965] | | 15 | Independent Site Test – AI Adapter + TI-RADS Prediction Operating Point | Sensitivity 0.751 [0.561, 0.942] Specificity 0.588 [0.518, 0.657] | Sensitivity 0.750 [0.593, 0.907] Specificity 0.624 [0.579, 0.668] | {37} 34 | 16 | Independent Site Test – TI-RADS Prediction AUC | Non-Parametric AUC 0.824 [0.725, 0.923] Parametric AUC 0.838 [0.745, 0.931] | Non-Parametric AUC 0.850 [0.758, 0.941] Parametric AUC 0.855 [0.759, 0.951] | | --- | --- | --- | --- | | 17 | Independent Site Test – TI-RADS Prediction Operating Point | Sensitivity 0.752 [0.561, 0.943] Specificity 0.405 [0.340, 0.470] | Sensitivity 0.821 [0.685, 0.958] Specificity 0.398 [0.350, 0.446] | | 18 | Measured Shape Test – Diagnostic Performance | Aspect 1.0 AUC 0.786 Aspect 1.0 Sensitivity 0.640 Aspect 1.0 Specificity 0.588 Aspect 1.1 AUC 0.792 Aspect 1.1 Sensitivity 0.640 Aspect 1.1 Specificity 0.615 Aspect 1.2 AUC 0.786 Aspect 1.2 Sensitivity 0.630 Aspect 1.2 Specificity 0.618 | Aspect 1.0 AUC 0.821 Aspect 1.0 Sensitivity 0.664 Aspect 1.0 Specificity 0.627 Aspect 1.1 AUC 0.828 Aspect 1.1 Sensitivity 0.655 Aspect 1.1 Specificity 0.654 Aspect 1.2 AUC 0.829 Aspect 1.2 Sensitivity 0.646 Aspect 1.2 Specificity 0.665 | | 19 | Subgroup Analysis: Age (AUC) | N/A | System AUC (Age <40): 0.859 [0.788, 0.930] AUC (Age 40-60): 0.834 [0.771, 0.898] | {38} 35 | | | | AUC (Age 60+): 0.836 [0.761, 0.911] Physicians AUC (Age <40): 0.759 [0.676, 0.842] AUC (Age 40-60): 0.728 [0.670, 0.786] AUC (Age 60+): 0.722 [0.668, 0.776] | | --- | --- | --- | --- | | 20 | Subgroup Analysis: Sex (AUC) | N/A | System AUC (Male): 0.806 [0.716, 0.896] AUC (Female): 0.848 [0.803, 0.892] Physicians AUC (Male): 0.704 [0.627, 0.780] AUC (Female): 0.749 [0.699, 0.799] | | 21 | Subgroup Analysis: Ethnicity (AUC) | N/A | System AUC (White): 0.823 [0.757, 0.889] AUC (Black): 0.726 [0.483, 0.969] AUC (Asian): 0.906 [0.780, 1.000] AUC (Hispanic): 0.807 [0.583, 1.000] | {39} 36 | | | | AUC (Other): 0.761 [0.557, 0.965] Physicians AUC (White): 0.721 [0.666, 0.776] AUC (Black): 0.673 [0.514, 0.831] AUC (Asian): 0.772 [0.661, 0.883] AUC (Hispanic): 0.788 [0.697, 0.879] AUC (Other): 0.663 [0.581, 0.745] | | --- | --- | --- | --- | | 22 | Subgroup Analysis: Age (Operating Point) | N/A | System Sensitivity (Age <40): 0.692 [0.545, 0.839] Sensitivity (Age 40-60): 0.714 [0.594, 0.835] Sensitivity (Age 60+): 0.686 [0.539, 0.832] Specificity (Age <40): 0.704 [0.603, 0.804] Specificity (Age 40-60): 0.639 [0.580, 0.697] Specificity (Age 60+): 0.586 [0.516, 0.656] Physicians Sensitivity (Age <40): 0.595 [0.378, 0.812] Sensitivity (Age 40-60): 0.606 [0.469, 0.743] | {40} 37 | | | | Sensitivity (Age 60+): 0.629 [0.449, 0.808] Specificity (Age <40): 0.400 [0.229, 0.571] Specificity (Age 40-60): 0.411 [0.290, 0.533] Specificity (Age 60+): 0.374 [0.259, 0.489] | | --- | --- | --- | --- | | 23 | Subgroup Analysis: Sex (Operating Point) | N/A | System Sensitivity (Male): 0.750 [0.610, 0.890] Sensitivity (Female): 0.677 [0.584, 0.771] Specificity (Male): 0.573 [0.473, 0.673] Specificity (Female): 0.640 [0.590, 0.689] Physicians Sensitivity (Male): 0.650 [0.466, 0.834] Sensitivity (Female): 0.589 [0.458, 0.720] Specificity (Male): 0.335 [0.207, 0.464] Specificity (Female): 0.406 [0.295, 0.518] | | 24 | Subgroup Analysis: Ethnicity (Operating Point) | N/A | System Sensitivity (White): 0.644 [0.534, 0.753] Sensitivity (Black): 1.000 [0.594, | {41} 38 | | | | 1.000] Sensitivity (Asian): 0.556 [0.190, 0.921] Sensitivity (Hispanic): 1.000 [0.862, 1.000] Sensitivity (Other): 0.700 [0.412, 0.988] Specificity (White): 0.601 [0.546, 0.656] Specificity (Black): 0.729 [0.606, 0.852] Specificity (Asian): 0.692 [0.445, 0.939] Specificity (Hispanic): 0.467 [0.204, 0.729] Specificity (Other): 0.509 [0.383, 0.635] Physicians Sensitivity (White): 0.578 [0.436, 0.720] Sensitivity (Black): 1.000 [0.616, 1.000] Sensitivity (Asian): 0.511 [0.178, 0.844] Sensitivity (Hispanic): 0.760 [0.268, 1.000] Sensitivity (Other): 0.640 [0.274, 1.000] Specificity (White): 0.392 [0.277, 0.507] Specificity (Black): 0.323 [0.181, 0.465] Specificity (Asian): 0.431 [0.095, 0.766] | | --- | --- | --- | --- | {42} 39 | | | | Specificity (Hispanic): 0.462 [0.093, 0.832]Specificity (Other): 0.363 [0.220, 0.505] | | --- | --- | --- | --- | Bench testing included verification of standalone performance, performance with TI-RADS and ATA outputs, as well as performance when compared to a separate data set including data from independent sites (separate and apart from the sites/data used to train and tune the algorithm). Testing demonstrates that application of the Koios DS AI Adapter exceeds physician performance as measured by AUC, sensitivity, and specificity. Descriptor predictions were tested objectively – against ground truth pathology. Testing demonstrated that performance requirements were met under ACR TI-RADS and ATA reporting systems as well as when compared against independent site data. Outputs were additionally tested subjectively and met the requirements for agreement with readers' descriptor categorizations. Testing characterized the sensitivity of the system with respect to shifts in the region of interest and variation in performance between high and low transducer frequencies. System performance on data acquired from independent sites meets performance requirements. Additionally, subgroup analysis of age, sex, and ethnicity returned results within the expected ranges. In all tests, the Thyroid engine met or exceeded performance requirements. {43} 40 ## Thyroid Smart Click A dataset of 650 nodules with corresponding physician-drawn ROI's were used for testing the Koios DS Thyroid Smart Click Engine. These ROI's are the same as the reference ROI's used to validate the Koios DS Thyroid Engine. A user "click" will be simulated for testing the performance of the Smart Click engine by calculating the center of each nodule. Non-inferiority testing is used to demonstrate that the use of the Smart Click engine does not degrade diagnostic performance when compared to physician-selected calipers. Each test will demonstrate that the lower 95% confidence bound of the difference in performance falls above a designated equivalence margin, delta (δ). In each case, delta is defined using the measured uncertainty in our performance metric and the variability observed on physicians' assessment of the data. Measurement uncertainty is computed via bootstrapping. Testing further provides quantitative metrics which demonstrate how closely the automated Smart Click ROIs match the manual physician-drawn ROI's using standard segmentation evaluation metrics. In this case, the Dice Similarity Coefficient will be used. Specifically, the test will measure and report the average DICE score between the Smart Click and physician ROI's across all of the images present in the validation set. The objective of this test is to demonstrate the similarity between Smart Click ROI's and manually drawn ROI's. Finally, testing demonstrates concretely that descriptors generated from Smart Click ROI's are not impacted by differences between them and manually drawn ROI's. Non-inferiority testing was used on a per-descriptor basis. Specifically, testing will show that the rate of agreement between the system's descriptors and the physicians' descriptors when utilizing smart click is non-inferior to the system's descriptors when using the manual ROI's. The Cohen's Kappa metric will be used to characterize agreement between system and reader in either case. Superiority is evaluated using similar methodology. In this case, the lower bound of the difference in performance must fall above zero. | Koios DS Smart Click Engine Test | Smart Click Engine Version (TOR.2.0.X) | | --- | --- | | 1: Non-inferiority Test - Sensitivity / Specificity | Sensitivity: | {44} 41 | | Difference = -0.021 [-0.046, 0.003] Result: Non-inferior Specificity: Difference = -0.001 [-0.024, 0.022] Result: Non-inferior | | --- | --- | | 2: Non-inferiority Test - AUC | Difference = -0.012 [-0.024, 0.000] Result: Non-inferior | | 3: Sub-optimal ROI Test | Difference = 0.013 [-0.014, 0.040] Result: Non-inferior | | 4: Detection DICE Coefficient | DICE= 0.913 +/- 0.075 | {45} 42 | 5: Non-inferiority Test - Descriptor Agreement | Composition: Difference = 0.003 [-0.020, 0.027] Result: Non-inferior Echogenicity: Difference = -0.003 [-0.016, 0.010] Result: Non-inferior Shape: Difference = -0.001 [-0.031, 0.029] Result: Non-inferior Margin: Difference = -0.004 [-0.017, 0.008] Result: Non-inferior Echogenic Foci: • Large Comet-Tail Artifacts | | --- | --- | {46} 43 | | Difference = -0.019 [-0.048, 0.010]Result: Non-inferiorMacrocalcificationsDifference = 0.017 [-0.035, 0.068]Result: Non-inferiorPeripheral (Rim) CalcificationsDifference = 0.024 [-0.072, 0.120]Result: Non-inferiorPunctate Echogenic FociDifference = 0.007 [-0.038, 0.053]Result: Non-inferior | | --- | --- | Testing demonstrates that system performance does not fall below the computed value for the equivalence margin, delta. The confidence interval of the difference falls within the expected bounds. The high value (DICE = 0.913 +/- 0.075) of the DICE coefficient demonstrates that Smart Click ROIs are, on average, a precise approximation to the ROIs that a physician would select. This in turn is a good indication that, as was the case with the overall diagnostic performance of the thyroid engine, the performance of the individual descriptors contained therein do not degrade as a result of using these ROIs. To further demonstrate this point, Test 5 contains a quantitative per-descriptor comparison of the predictions generated when using Smart Click ROIs to those generated using physician ROIs, via Non-Inferiority testing. Noninferiority was demonstrated for each descriptor using Smart Click ROIs. Together, these results clearly demonstrate descriptor performance is not negatively impacted by the use of the Smart Click engine. Image Registration and Matching {47} 44 A dataset consisting of 1,600 ultrasound studies of lesions in both breast (950 cases) and thyroid (650 cases) was used to evaluate the performance of the Koios DS Client ROI Match Engine in Tests 1-4 outlined below. Each study consists of one or more ultrasound images, wherein one or more contain a region of interest (ROI) that signifies the location of a lesion within the image. Tests 5 & 6 utilized the breast validation dataset of 1014 cases. Tests 7 & 8 utilized the thyroid validation dataset of 650 cases. Tests 5 & 7 measure the rate of incidence of each of the possible outcomes of the matching process. - Successful Match: The system correctly identifies the image and image region corresponding to the query screenshot. - No Match: The system is unable to identify a matching region of interest and returns no match for the query screenshot. Note: This is considered a positive outcome. If the system identifies that it cannot match a query screenshot, it should return that status rather than an incorrect result. - Incorrect Match: The system identifies the correct image but returns an incorrect region within that image. Incorrect is defined as under 0.5 intersect-over-union with respect to the correct region. - Incorrect Image: The system selects the wrong image as a match for the query screenshot. These tests were run on each of the ROI's contained in the test dataset, which corresponds to 2028 breast ROI's and 1288 thyroid ROI's. These tests are to tabulate how often the image matching process results in each of the four possible outcomes. This is reported both by total count and percent incidence. Tests 6 & 8 investigate the quality of the resulting registration. This is measured by computing the average DICE coefficient for the set of matches which are in the successful match category. A summary of performance statistics is outlined below. | Koios DS Image Registration and Matching Engine Test | Image Registration and Matching Engine Version (LAM.1.X.X) | | --- | --- | {48} 45 | 1: No Match Rate | No Match Rate = 0.32% | | --- | --- | | 2: Match Time | Average Time for Study Preprocessing: 2.39 +/- 0.48 seconds Average Time for Image Matching: 0.22 +/- 0.12 seconds Note: The average study size which was processed in this experiment was 53 +/- 11 images. The measured durations scale roughly linearly with study size. | | 3: End-to-End Breast Engine Performance | AUC = 0.946 Sensitivity = 0.975 Specificity = 0.637 | | 4: End-to-End Thyroid Engine Performance | AUC = 0.838 Sensitivity = 0.700 Specificity = 0.637 | {49} 46 | 5: Breast Image Matching Outcomes | Successful Match: Count: 2018 Fraction: 0.995 No Match: Count: 10 Fraction: 0.005 Incorrect Match: Count: 0 Fraction: 0.000 Incorrect Image: Count: 0 Fraction: 0.000 | | --- | --- | | 6: Breast Image Matching DICE Coefficient | DICE = 0.995 =/- 0.005 | {50} 47 | 7: Thyroid Image Matching Outcomes | Successful Match: Count: 1288 Fraction: 1.000 No Match: Count: 0 Fraction: 0.000 Incorrect Match: Count: 0 Fraction: 0.000 Incorrect Image: Count: 0 Fraction: 0.000 | | --- | --- | | 8: Thyroid Image Matching DICE Coefficient | DICE = 0.996 =/- 0.004 | **Auto-populate Size and Position using Optical Character Recognition (OCR)** {51} 48 A dataset of 1910 ultrasound B-Scans was manually annotated to test the OCR engine. These are a mix of thyroid and breast images that come from a variety of machines. Of these, a subset of 1226 images that come from the supported list of machines is used to carry out this test. Tests will measure accuracy (percent correct) for each of the structured fields predicted by the engine. False positive, false negative, and misread text fields will count against the accuracy measurement of the engine. | Koios DS OCR Engine Test | Optical Character Recognition (OCR) Engine Version (GNO.1.1.X) | | --- | --- | | 1: Breast Freetext Identification | Breast Side: 0.983 Location Type: 0.948 Clock Hour: 0.926 Clock Minute: 0.934 CMFN: 0.944 Plane: 0.976 | | 2: Thyroid Freetext Identification | Thyroid Side: 0.965 Pole: 0.976 Region: 0.998 Plane: 0.970 | {52} 49 | 3: Measurement Text Identification | Measurement Description: 0.943 Measurement Value: 0.948 Unit of Measurement: 0.967 | | --- | --- | In conclusion, the subject device has demonstrated substantially equivalent performance to the predicate by showing statistically significant results against similar success criteria in bench testing comparisons. ## 9. Performance Testing – Clinical ### Breast A clinical study was previously executed to determine the effect of Koios DS Breast (K190442) on reader performance. There were no new clinical studies performed for the design and development of Koios DS 3.0 (K212616), which used Koios DS Breast (K190442) as a predicate device during its regulatory submission. As discussed in the prior section, the performance of Koios DS 3.0 has been met or significantly improved across all measured metrics by Koios DS 3.6. This data continues to apply to the breast functionality within the subject device, with the understanding that its performance is superior, and it would therefore provide an equivalent or greater benefit. The below summary of the clinical study data has been included for ease of reference. The study objective was to determine the impact on Interpreting Physician (Reader) performance as defined by the area under the Receiver Operating Characteristic (ROC) Curve (AUC) when Koios DS Breast and an ultrasound examination are combined (USE + DS), compared to USE Alone in patients that present with a soft tissue breast lesion through any form of imaging or physical examination and are referred for diagnostic ultrasound. {53} 50 The study consisted of 15 readers with varying levels of training and experience providing analysis on a randomized set of 900 patient cases presented with USE + DS and USE Alone in two reading periods separated by a 1-month wash-out, totaling 1800 cases analyzed per reader. The reader set and dataset were distributed in accordance with FDA guidance and are explained in detail below: {54} 51 # Reader Background | Reader ID | Board Certification/ Specialty | Breast Fellowship Trained and/or Dedicated Breast Imager | Years of Experience – Mammography and/or Breast Ultrasound | Academic Institution Affiliation (Yes/No) | MQSA Qualified Interpreting Physician | | --- | --- | --- | --- | --- | --- | | 1 | Diagnostic Radiology | No | 13 years | No | Yes | | 2 | Diagnostic Radiology | No | 4 years | No | No | | 3 | Diagnostic Radiology | Yes | 7 years | Yes | Yes | | 4 | Breast Surgeon | No | 0 years | No | No | | 5 | OB/GYN | No | 20 years | No | No | | 6 | Diagnostic Radiology | No | 13 years | Yes | No | | 7 | Diagnostic Radiology | No | 3 years | Yes | No | | 8 | OB/GYN | No | 0 years | No | No | | 9 | Diagnostic Radiology | Yes | 15 years | No | Yes | | 10 | Diagnostic Radiology | No | 13 years | No | No | | 11 | Diagnostic Radiology | Yes | 30 years | No | Yes | | 12 | Diagnostic Radiology | Yes | 10 years | Yes | Yes | | 13 | Diagnostic Radiology | No | 0 years | No | No | | 14 | Interventional Radiology | No | 4 years | No | No | | 15 | Breast Surgeon | No | 25 years | Yes | No | {55} 52 # **Dataset Demographic Information** The Koios DS Breast engine was tested on images sourced from a wide variety of ultrasound hardware and data with the following patient demographics to ensure the system performance is generalizable to and representative of diverse populations. Patient demographic distribution was based upon data from the Breast Cancer Surveillance Consortium (2006-2009)². The following figures represent the final validation dataset (900 cases): ² Data were obtained from the Breast Cancer Surveillance Consortium, funded by the National Cancer Institute (HHSN261201100031C). From the Breast Cancer Surveillance Consortium website, http://www.bcsc-research.org/ {56} 53  Distribution of Malignancy in Final Validation Set  Distribution of BI-RADS Category in Final Validation Set  Distribution of Ethnicity in Final Validation Set  Distribution of Age in Final Validation Set {57} 54  Distribution of Lesion Size in Final Validation Set  Distribution of Invasive Cancer in Final Validation Set  Distribution of BI-RADS Density in Final Validation Set {58} 55  Per the primary endpoint of the study, ROC curves were generated and analyzed. All AUCs were computed via the trapezoidal approximation. Based on the standard error measurements, the error can be propagated to estimate the mean performance interface and 95% confidence interval. This was found to be 0.0370 (0.030, 0.044) at α = .05, satisfying the success criteria for the primary endpoint. To characterize the effect of Koios DS (USE + DS) system on inter-operator variability, the Kendall Tau-B correlation coefficient was computed in a pairwise manner for all readers. The metric is > 0 for all reader pairs. The standard error for USE + DS and USE Alone was computed to assess if the shifts in the metric were significant. The average Kendall Tau-B of USE Alone was .5404 (.5301, .5507) and the average Kendall Tau-B of USE + DS was .6797 (.6653, .6941) with 95% CI demonstrating a significant increase in the metric (α = .05). Also assessed was the effect of Koios DS on intra-operator variability leveraging 150 reads that did not switch from USE Alone to USE + DS across the washout session in the reader study (75 each). USE Alone class switching rate was 13.6% and the USE + DS class switching rate was 10.8% (p = 0.042), demonstrating a statistically significant reduction in intra-reader variability when using USE + DS. {59} 56 ## Thyroid An observational case-controlled, Multi-Reader, Multi-Case (MRMC) retrospective clinical trial (CRRS-3) was executed to determine the effect of Koios DS Thyroid on reader performance. Effect on performance was defined by measuring the area under the Receiver Operating Characteristic (ROC) Curve (AUC) when Koios DS and an ultrasound examination were combined (USE + DS), compared to unassisted TI-RADS based Reader performance (USE Alone). All data analysis cases consisted of USE Alone and USE + DS image readings in patients that presented with a thyroid abnormality through any form of imaging or physical examination and were referred for diagnostic ultrasound where a nodule was subsequently discovered. Data analysis in the CRRS-3 study was based on 650 retrospectively collected cases that were assigned a TI-RADS Assessment Category 1 through 5 at the time of initial review at study entry based upon the interpreting physician of the ultrasound evaluation. The study consisted of 15 readers reviewing and interpreting 650 cases twice (1300 total cases per reader). All data analysis was based on two randomized evaluations of each case with and without the assistance of Koios DS software with a 1-month washout period between corresponding presentations of the case and interpretations by physicians. The study design called for a mixed population of physician readers (11/15 or 73% US based) and cases (500 or 77% US based) coming from both the US and Europe. Readers with a current medical license who met inclusion criteria and completed the study training protocol were considered trained interpreting physicians for study purposes. Readers possessed varying levels of training and experience, as detailed below: ### Reader Experience | Reader ID | Reader Category | Experience (post-residency) | | --- | --- | --- | | R1 | Domestic Endocrinologist (End) | < 10 years | | R2 | Domestic Radiologist (Rad) | ≥ 20 years | | R3 | Domestic Rad | ≥ 20 years | | R4 | Domestic Rad | ≥ 10 and < 20 years | | R5 | Domestic Rad | ≥ 10 and < 20 years | | R6 | Domestic Rad | ≥ 10 and < 20 | | R7 | Domestic Rad | ≥ 20 years | | R8 | Domestic Rad | < 10 years | | R9 | Domestic Rad | ≥ 20 years | {60} 57 | R10 | Domestic Rad | ≥ 20 years | | --- | --- | --- | | R11 | Domestic End | < 10 years | | R12 | European Rad | ≥ 20 years | | R13 | European Rad | ≥ 20 years | | R14 | European End | ≥ 20 years | | R15 | European End | ≥ 20 years | ### Dataset Demographic Information The Koios DS thyroid engine was tested on images sourced from a wide variety of ultrasound hardware and data with the following patient demographics to ensure the system performance is generalizable to and representative of diverse populations. The following ultrasound hardware represents the final validation dataset (650 cases). {61} 58  {62} 59 The final validation set data is divided into 2 subsets; 500 cases from United States locations and 150 cases from European locations. The following figures represent the United States patient demographics: {63} 60  Distribution of Malignancy in the Final Validation Set (United States)  Distribution of TI-RADS Assessment in the Final Validation Set, (United States)  Distribution of Patient Ethnicity in the Final Validation Set (United States)  Distribution of Patient Sex in the Final Validation Set (United States) {64} 61  *Distribution of Patient Age in the Final Validation Set (United States)* {65} 62 The following figures represent the European patient demographics:   *Distribution of Patient Sex in the Final Validation Set (European)* {66} 63 The primary CRRS-3 analysis was performed on the Readers’ TI-RADS point total gradings from their review of the USE Alone and their review of the USE + DS for the Non-Cancer Case Set and Cancer Case Set. For each Reader, two ROC curves (Sensitivity vs. 1 – Specificity) were plotted using the USE Alone and the USE + DS primary analysis cases. Reader-specific AUC values for the primary analysis were derived from the trapezoidal approximation, whereas the mean AUC values and associated standard errors within- and between-modality across all Readers were derived from the DBM (Dorfman-Berbaum-Metz ANOVA after jackknife) method. This approach captures both reader variability and case variability and is the standard methodology for comparing AUCs in MRMC studies. All ROC curve analysis was done with respect to cyto-/histological or excisional pathology. Summary of All Primary Study Endpoints and Secondary Analyses (US data in bold) | Analysis | Overview | Result | | --- | --- | --- | | Primary Endpoint 1 | Change in average AUC with Koios DS (all readers, all data) | +0.083 [0.066, 0.099] (parametric) +0.079 [0.062, 0.096] (non-parametric) | | Primary Endpoint 2 | Change in average AUC with Koios DS (US readers, US data) | +0.074 [0.051, 0.098] (parametric) +0.073 [0.049, 0.096] (non-parametric) | | Secondary Analysis 1 | Change in average Sensitivity and Specificity of FNA with Koios DS (all readers, all data) | + 0.084 [0.054, 0.113] (sensitivity) + 0.140 [0.125, 0.155] (specificity) | | | Change in average Sensitivity and Specificity of FNA with Koios DS (US readers, US data) | + 0.058 [0.017, 0.098] (sensitivity) + 0.130 [0.110, 0.151] (specificity) | | | Change in average Sensitivity and Specificity of FNA with Koios DS (EU readers, EU data) | +0.125 [0.014, 0.237] (sensitivity) +0.171 [0.109, 0.233] (specificity) | {67} 64 | Secondary Analysis 2 - – excluding cases recommended for FNA | Change in average Sensitivity and Specificity of Follow-up with Koios DS (all readers, all data) | + 0.092 [0.043, 0.141] (sensitivity) + 0.242 [0.220, 0.264] (specificity) | | --- | --- | --- | | | Change in average Sensitivity and Specificity of Follow-up with Koios DS (US readers, US data) | + 0.087 [0.023, 0.151] (sensitivity) + 0.206 [0.176, 0.235] (specificity) | | | Change in average Sensitivity and Specificity of Follow-up with Koios DS (EU readers, EU data) | +0.084 [-0.133, 0.300] (sensitivity) +0.350 [0.267, 0.434] (specificity) | | Secondary Analysis 2a – including cases recommended for FNA | Change in average Sensitivity and Specificity of Follow-up with Koios DS (all readers, all data) | +0.060 [0.040, 0.080] (sensitivity) +0.206 [0.192, 0.219] (specificity) | | | Change in average Sensitivity and Specificity of Follow-up with Koios DS (US readers, US data) | +0.053 [0.026, 0.080] (sensitivity) +0.180 [0.161, 0.198] (specificity) | | | Change in average Sensitivity and Specificity of Follow-up with Koios DS (EU readers, EU data) | +0.060 [-0.009, 0.129] (sensitivity) +0.296 [0.238, 0.354] (specificity) | | Secondary Analysis 3 | Change in average AUC with Koios DS (EU Readers, EU Data) | + 0.079 [0.024, 0.134] (parametric) + 0.066 [0.014, 0.118] (non-parametric) | {68} 65 | Secondary Analysis 4 | Inter-Reader Variability measuring the association of TI-RADS points assigned with and without decision support Difference (Relative Change %) | 40.7% (all readers, all data) 37.4% (US readers, US data) 49.7% (EU Readers, EU Data) | | --- | --- | --- | | Secondary Analysis 5 | Impact on Interpretation Time | -23.6% (all readers, all data) -22.7% (US readers, US data) -32.4% (EU Readers, EU Data) | | Secondary Analysis 6 | Change in average AUC with Koios DS descriptor classifiers only (without AI Adapter) (parametric) | +0.022 [0.005, 0.039] (all readers, all data) +0.017 [-0.007, 0.041] (US readers, US data) +0.010 [-0.051, 0.071] (EU Readers, EU Data) | | | Change in average AUC with Koios DS descriptor classifiers only (without AI Adapter) (non- parametric) | +0.019 [0.001, 0.037] (all readers, all data) +0.015 [-0.010, 0.039] (US readers, US data) +0.004 [-0.054, 0.062] (EU Readers, EU Data) | | | Change in average sensitivity and specificity of FNA with Koios DS descriptor classifiers only (without AI Adapter) | Sensitivity: +0.052 [0.022, 0.081] (all readers, all data) +0.026 [-0.014, 0.066] | {69} 66 | | | (US readers, US data)+0.109 [-0.004, 0.221](EU Readers, EU Data)Specificity-0.009 [-0.024, 0.006](all readers, all data)-0.001 [-0.02…