Cris Benge

other information

1. 

   Honors & Awards

   Honors and awards from my adventures through life and work that I am most proud of.


2. 

   Skills & Certifications

   Key competencies and an unreasonbly long list of professional certifications.


3. 

   Education

   Academic accomplishments & pursuits.
   
   
4. Research

   BERTVision: A Parameter-Efficient Approach for Question Answering

   Siduo Jiang, Cris Benge, William Casey King

   Improving span annotation and classification task performance using parameter-efficient model architectures trained on BERT's hidden-state activations. 2022

   Abstract Paper Code Website

   We present a highly parameter-efficient approach for Question Answering (QA) that significantly reduces the need for extended BERT fine-tuning. Our method uses information from the hidden state activations of each BERT transformer layer, which is discarded during typical BERT inference. Our best model achieves maximal BERT performance at a fraction of the training time and GPU/TPU expense. Performance is further improved by ensembling our model with BERT’s predictions. Furthermore, we find that near optimal performance can be achieved for QA span annotation using less training data. Our experiments show that this approach works well not only for span annotation, but also for classification, suggesting that it may be extensible to a wider range of tasks.

5. Research

   A High Performance Compression Approach for Transformer-Based NLP Tasks

   Siduo Jiang, Cris Benge, Andrew Fogarty, William Casey King, Alberto Todeschini, Hossein Vahabi

   Improving performance of a wide variety of NLP tasks using parameter-efficient model architectures trained on BERT's hidden-state activations. 2021

   Abstract Paper Code Website

   We present a new BERT compression technique for NLP tasks that significantly reduces the number of parameters used by the model. Our compression method uses information from the hidden state activations of each BERT transformer layer, which is discarded during typical BERT inference. We achieve same accuracy as BERT across a wide variety of GLUE benchmark tasks and SQuAD 2.0 with 209x times less parameters needed. This suggests that the method may be extensible to a wider range of NLP tasks.

6. Research

   Ticket-BERT: Labeling Incident Management Tickets with Language Models

   Zhexiong Liu, Cris Benge, Siduo Jiang

   Enabling automation of trouble ticket classification from a diverse set of machine and human-generated texts. 2022

   Abstract Paper

   Software and hardware incident management has gained notable attention in enterprise-level products, where efficient ticket resolution services are critically needed due to large-scale ticket resolution demands. An essential aspect of prioritizing incident tickets for resolution is efficiently labeling tickets with fine-grained categories. This facilitates the accurate triage of incidents to responsible teams and ultimately accelerates ticket resolution. However, ticket data is often complex and poses several unique challenges for modern machine learning methods: (1) tickets are created and updated either by machines with pre-defined algorithms or by engineers with domain expertise that share different protocols, (2) tickets receive frequent revisions that update ticket status by modifying all or parts of ticket descriptions, and (3) ticket labeling is time-sensitive and requires knowledge updates and new labels per the rapid software and hardware improvement lifecycle. To handle these issues, we introduce Ticket-BERT, which trains a simple yet robust language model for labeling tickets using the proposed ticket datasets (i.e., D-Human, D-Machine, D-Mixture). Our extensive experiments demonstrate the superiority of Ticket-BERT over a diverse set of baselines and state-of-the-art text classifiers in Microsoft Cognitive Services. We further evaluate Ticket-BERT using human input on a set of hard-to-identify tickets and achieve outstanding performance. We deploy the proposed models in an active learning cycle that makes the model adapt to newly collected data and achieve greater accuracy with little annotating burden.

7. Research

   STIM: Predicting Memory Uncorrectable Errors with Spatio-Temporal Transformer

   Zhexiong Liu, Cris Benge, Siduo Jiang

   Predicting failure of memory due to uncorrectable errors using spatio-temporal transformers. 2023

   Abstract Paper

   Memory Uncorrectable Errors (UEs) have been identified as a leading cause of server crashes in Microsoft Azure datacenters. To mitigate this, leveraging machine learning methods to predict UEs before undertaking preventive maintenance has become an effective measurement to decrease server downtime in large-scale clusters. However, the prediction of UEs presents several challenges: (1) anomaly UEs are exceedingly rare in memory events. This rarity renders machine learning methods highly vulnerable to imbalanced data, leading to high false positives and low recall rates. (2) memory data is inherently noisy and heterogeneous because memory hardware comes from a range of manufacturers (e.g., Samsung, SK-Hynix) and is deployed in diverse operational environments. This necessitates the need for robust machine learning models. (3) Predicting UEs demands the handling of intricate spatial and temporal variability in the memory framework, given the hardware degradation over its lifecycle and the iterative introduction of new software and hardware configurations. To address these issues, we proposed a Spatial-temporal Transformer in Memory (STIM) model that learns spatial and temporal features with self-attention mechanisms across memory cells. Comparative evaluations on our developed Azure Memory Error Dataset (AMED) reveal that our proposed STIM model outperforms existing machine learning methods and strong baselines, which marks the significant application of Language Models (LMs) to address complex UE prediction tasks.