Representation learning for time-domain high-energy astrophysics: Discovery of extragalactic fast X-ray transient XRT 200515

Abstract: We present a novel representation learning method for downstream tasks like anomaly detection, unsupervised classification, and similarity searches in high-energy data sets. This enabled the discovery of a new extragalactic fast X-ray transient (FXT) in Chandra archival data, XRT 200515, a needle-in-the-haystack event and the first Chandra FXT of its kind. Recent serendipitous discoveries in X-ray astronomy, including FXTs from binary neutron star mergers and an extragalactic planetary transit candidate, highlight the need for systematic transient searches in X-ray archives. We introduce new event file representations, E-t maps and E-t-dt cubes, that effectively encode both temporal and spectral information, enabling the seamless application of machine learning to variable-length event file time series. Our unsupervised learning approach employs PCA or sparse autoencoders to extract low-dimensional, informative features from these data representations, followed by clustering in the embedding space with DBSCAN. New transients are identified within transient-dominant clusters or through nearest-neighbour searches around known transients, producing a catalogue of 3559 candidates (3447 flares and 112 dips). XRT 200515 exhibits unique temporal and spectral variability, including an intense, hard <10s initial burst, followed by spectral softening in an ~800s oscillating tail. We interpret XRT 200515 as either the first giant magnetar flare observed at low X-ray energies or the first extragalactic Type I X-ray burst from a faint, previously unknown low-mass X-ray binary in the LMC. Our method extends to data sets from other observatories such as XMM-Newton, Swift-XRT, eROSITA, Einstein Probe, and upcoming missions like AXIS.

All the data is available on Huggingface at https://huggingface.co/datasets/StevenDillmann/chandra_xray_eventfiles. Note that the eventfiles_table.csv already includes preprocessed eventfiles. You can filter for a specific eventfile with the obsreg_id column. The eventfile is then just all rows that are labeled with this obsreg_id ordered by time.

\begin{document}
\label{firstpage}
\pagerange{\pageref{firstpage}--\pageref{lastpage}}

\begin{abstract}
We present a novel representation learning method for downstream tasks like anomaly detection or unsupervised classification in high-energy datasets. This enabled the discovery of a new extragalactic fast X-ray transient (FXT) in \emph{Chandra} archival data, XRT\,200515, a needle-in-the-haystack event and the first \emph{Chandra} FXT of its kind. Recent serendipitous discoveries in X-ray astronomy, including FXTs from binary neutron star mergers and an extragalactic planetary transit candidate, highlight the need for systematic transient searches in X-ray archives. We introduce new event file representations, $E-t$\,Maps and $E-t-dt$\,Cubes, that effectively encode both temporal and spectral information, enabling the seamless application of machine learning to variable-length event file time series. Our unsupervised learning approach employs PCA or sparse autoencoders to extract low-dimensional, informative features from these data representations, followed by clustering in the embedding space with DBSCAN. New transients are identified within transient-dominant clusters or through nearest-neighbor searches around known transients, producing a catalog of 3,559 candidates (3,447 flares and 112 dips). XRT\,200515 exhibits unique temporal and spectral variability, including an intense, hard $<$10\,s initial burst, followed by spectral softening in an $\sim$800\,s oscillating tail. We interpret XRT\,200515 as either the first giant magnetar flare observed at low X-ray energies or the first extragalactic Type I X-ray burst from a faint, previously unknown LMXB in the LMC. Our method extends to datasets from other observatories such as \emph{XMM-Newton}, \emph{Swift-XRT}, \emph{eROSITA}, \emph{Einstein Probe}, and upcoming missions like \emph{AXIS}.
\end{abstract}

\begin{keywords}
software: machine learning, methods: data analysis, X-rays: bursts, stars: magnetars, transients: gamma-ray bursts, stars: peculiar
\end{keywords}

\section{Introduction}

Recent serendipitous discoveries, such as extragalactic fast X-ray transients (FXTs) linked to neutron star merger candidates as electromagnetic counterparts to gravitational wave events \citep{2022ApJ...927..211L} and an X-ray dip associated with the first extragalactic planet candidate \citep{2021NatAs...5.1297D}, underscore the challenges of identifying such rare events within large X-ray catalogs. Beyond magnetar-powered FXTs as the aftermath of binary neutron star mergers \citep{2006Sci...311.1127D, 2008MNRAS.385.1455M, 2013ApJ...763L..22Z, 2017ApJ...835....7S, 2017MNRAS.467.4841B, 2019Natur.568..198X}, other interesting origins of extragalactic FXTs include supernova shock breakouts (SBOs) \citep{2008Natur.453..469S, 2009ApJ...702..226M, 2020ApJ...896...39A, 2020ApJ...898...37N}, tidal disruption events (TDEs) \citep{2013ApJ...779...14J} including quasi-periodic eruptions (QPEs) \citep{2021Natur.592..704A, 2021ApJ...921L..40C}, or binary self-lensing events \citep{2018MNRAS.474.2975D, 2020MNRAS.491.1506D, 2020MNRAS.495.4061H}. Both because of their very stochastic nature, and because narrow-field X-ray missions such as the Chandra X-ray Observatory (\emph{Chandra}) \citep{2000SPIE.4012....2W}, \emph{XMM-Newton} \citep{2001A&A...365L...1J} and \emph{Swift-XRT} \citep{2005SSRv..120..165B} are not designed as wide time-domain surveys, X-ray transient discoveries are often serendipitous. They can be found in observations that were originally proposed for a completely unrelated science objective and are rarely the target of the observation. In many cases serendipitously found X-ray sources do not get characterized or classified, since their transient nature is not immediately obvious. Instead, observations with X-ray transients often get stored in large data archives and remain unnoticed. This raises the need for a systematic search for short-duration phenomena in high-energy catalogs. New missions such as \emph{eROSITA} \citep{2021A&A...647A...1P}, \emph{Einstein Probe} \citep{2022hxga.book...86Y} and the upcoming \emph{AXIS} Observatory \citep{2024SPIE13093E..28R} target X-ray transients more directly, thus the development of novel transient detection methods is becoming even more relevant. The temporary, unpredictable and `unusual' nature of X-ray transients distinguishes them from `normal' X-ray source emissions. From a data science perspective, they can be understood as `anomalies' within a large dataset. Existing methods for identifying X-ray transients primarily rely on statistical tests of variability \citep{2019MNRAS.487.4721Y, 2020A&A...640A.124P, quirola2022extragalactic, 2023arXiv230413795Q}. While effective within specific constraints, these approaches are inherently limited by their underlying assumptions, which may not capture the diverse nature of transient phenomena. In contrast, machine learning offers a more flexible, expressive, and scalable framework, making it particularly well-suited for anomaly detection in large, high-dimensional datasets with diverse transient types. While optical time-domain surveys are at the forefront of leveraging extensive observational programs, like \emph{ZTF} \citep{2019PASP..131a8002B} or the upcoming \emph{LSST} survey \citep{2019ApJ...873..111I}, and neural network-based anomaly detection tools to identify rare sources among countless ordinary objects \citep{2021ApJS..255...24V,2021MNRAS.508.5734M,2022MNRAS.517..393M}, the X-ray astronomy community has only recently begun exploring the potential of machine learning to classify sources \citep{2022ApJ...941..104Y, 2024MNRAS.528.4852P} or to search for needle-in-a-haystack events in large X-ray datasets and archives \citep{2022A&A...659A..66K, sdthesis}. The effectiveness of machine learning methods largely depends on the algorithm's ability to learn useful representations from the data.

\emph{Representation learning} \citep{bengio2013representation} is an increasingly popular technique in astronomy used in supervised, semi-supervised, self-supervised and unsupervised frameworks \citep{2018NatAs...2..151N, 2021ApJ...911L..33H, 2022MNRAS.513.1581W, 2024RASTI...3...19S, 2024MNRAS.530.1274M}. It involves creating or learning meaningful representations for specific modalities of scientific data, which can then be used for downstream tasks such as regression, classification, or, as in this work, anomaly detection. The compressed representations live in a low-dimensional embedding space, in which anomalous data samples are well-separated from more ordinary ones.

We propose a new unsupervised representation learning method to perform a large-scale search for X-ray transients in the \emph{Chandra} archive. High-energy catalogs include individual X-ray source observations in the form of event files. The variable length of these time series poses a challenge in creating consistent representations suitable for transient searches with machine learning. Most deep learning algorithms take a fixed-length input for all data samples. In order to effectively represent event files over a broad range of lengths, we introduce novel fixed-length event file representations, which take into account both their time-domain and energy-domain information. Applying feature extraction and dimensionality reduction techniques, for example with sparse autoencoders, we create a representation space that encodes scientifically meaningful information, such as the spectral and variability properties of the astrophysical sources. Previously identified X-ray transients occupy distinct, well-isolated clusters in the embedding space. Using clustering techniques and nearest neighbor searches allows us to effectively explore these transient-dominant clusters to discover new X-ray transients. We collect the identified X-ray flare and dip candidates in a publicly available catalog, serving as a fertile ground for new discoveries in time-domain high-energy astrophysics.

Among these candidates, we identify an intriguing extragalactic FXT, XRT\,200515, which exhibits unique temporal and spectral characteristics distinct from any previously reported \emph{Chandra} FXTs. The transient's initial hard $<$10\,s burst shows a sharp rise exceeding 4 orders of magnitude, followed by spectral softening in an $\sim$800\,s oscillating tail. This transient is likely related to either a giant magnetar flare (GMF) from a distant soft gamma repeater (SGR) behind the Large Magellanic Cloud (LMC) or an extragalactic Type I X-ray burst from a faint LMXB in the LMC. Each of these interpretations presents its own set of challenges. Alternatively, XRT\,200515 could be a new type of astronomical phenomenon found by our anomaly detection method using machine learning.

Our method is the first unsupervised representation learning approach for anomaly detection in high-energy astrophysics. It is applicable to datasets from high-energy catalogs like \emph{Chandra}, \emph{XMM-Newton}, \emph{Swift-XRT}, \emph{eROSITA}, and \emph{Einstein Probe}. We created semantically meaningful representations that were also used for regression and classification \citep{sdthesis}. These can later be aligned with other data modalities, such as optical images or infrared spectra to design multi-modal models \citep{2024MNRAS.531.4990P, 2024arXiv240308851M, 2024arXiv240816829Z, 2024arXiv241108842R, themultimodaluniversecollaboration2024multimodaluniverseenablinglargescale} using contrastive learning \citep{pmlr-v139-radford21a}, that can improve on current state-of-the-art methods used to characterize the associated objects. Ultimately, this work and other representation and contrastive learning approaches lay the groundwork for developing large-scale foundation models in astronomy.

The paper is organized as follows: In \S~\ref{sec:data}, we provide information on the dataset of \emph{Chandra} event files used in this analysis. In \S~\ref{sec:methods}, we describe in detail the implementation of our novel transient detection approach leveraging representation learning. In \S~\ref{sec:results}, we present and discuss the results in form of the semantically meaningful representation space of the event files, the catalog of X-ray transient candidates and the discovery of the new \emph{Chandra} transient XRT\,200515. Finally, we highlight our contributions to time-domain high-energy astrophysics and outline potential directions for extending this work in the future in \S~\ref{sec:discussion}.

The relevant code, a demonstration of the pipeline, and an interactive embedding selection, transient search and lightcurve plotting tool are available online at the \texttt{GitHub} repository \href{https://github.com/StevenDillmann/ml-xraytransients-mnras}{https://github.com/StevenDillmann/ml-xraytransients-mnras}.

\section{Dataset}
\label{sec:data}

We use data from the Chandra Source Catalog (CSC) version 2.1 \citep{2024arXiv240710799E}, which includes all publicly available X-ray sources detected by \emph{Chandra} as of December 2021. For this study, we focus specifically on observations from the Advanced CCD Imaging Spectrometer (ACIS). CSC 2.1 had not been fully released at the time our analysis was performed, but catalog data was available for sources that had completed processing in the \emph{Current Database View}}, a snapshot of which we took on 11 April 2023. CSC 2.1 performs source detection on stacked observations, and catalog properties are provided both for these stack-level detections, and for each of observation-level detection that contribute to a stack detection. Because we are interested in short-time variability that happens within a \emph{single} observation of a source, we use the catalog products for the observation-level detections in our analysis. For a given X-ray detection, two types of products are provided in the CSC: (i) database tables with source properties, such as fluxes in the different X-ray energy bands, hardness ratios, variability indices, etc., and (ii) file-based data products for each detection of a source, such as the detect regions, the \emph{Chandra} PSF at that location, etc. The following observation-level catalog properties are relevant for our analysis:

\begin{itemize}
\item \verb+var_prob_b+: The probability that a source detection is variable in time for the broad energy band (0.5--7\,keV), as estimated using the Gregory-Loredo algorithm \citep{1992ApJ...398..146G}. In this paper we call this quantity $p_{\rm{var}}^b$.

\item \verb+var_index_b+: The variability index in the broad band, which indicates the level of confidence for time variability. A variability index of 6 or larger indicates variability at a confidence of at least 2$\sigma$. In this paper we call this quantity $I_{\rm{var}}^b$.

\item \verb+hard_<hs/ms/hm>+: The hardness ratios, which quantify the relative fraction of photons detected in two given bands chosen between the soft (0.5--1.2\,keV), medium (1.2--2\,keV), and hard (2--7\,keV) bands for a source detection. For example, a value of the hard-to-soft hardness ratio close to 1 indicates that most of the photons detected are in the hard energy band, whereas a value close to $-1$ indicates that most photons are detected in the soft band. In this paper we call these quantities $HR_{\rm{hs}}$, $HR_{\rm{ms}}$, and $HR_{\rm{hm}}$.

\end{itemize}

From the catalog data products available for observation-level X-ray detections, we are interested in the region event file. This event file consists of a list of all individual photon events detected in a small bounding box around a source detection, listing their energies, arrival times, and detector coordinates. These multivariate time series are the basis for the characterization of an X-ray source: lightcurves, spectra, images, coordinates, and other properties are derived from the distribution of the listed quantities. In this analysis, we directly use these event files as our primary data products. The values of the catalog properties listed above serve as summary statistics for the detection associated with a given region event file. We only include event files with more than 5 events and a signal-to-noise ratio above 5 to minimize spurious signals from low-number statistics in faint sources. We also exclude detections that are flagged for pile-up. Pileup leads to a decrease in the observed count rate and skews the spectrum towards higher energies \citep{davis2007pile}}, i.e., those with a pileup fraction larger than 5\%, which corresponds to a maximum pileup warning of 0.1 in CSC 2.1. For the resulting detections, we filter the event files to include only events contained within the detection region for each source. These detection regions are also provided as data products in CSC 2.1, and consist of the ellipse that includes the 90\% encircled counts fraction of the PSF at the source location. Due to the low background level in \emph{Chandra} observations, the majority of events selected after this spatial filtering are expected to be events associated with the X-ray source, not the background. In the selected event files, we only include photon events within good time intervals (GTIs), which are time periods of valid, high-quality data. No other pre-processing is required. The final dataset consists of 95,473 filtered event files from 58,932 sources, resulting in an average of $1.62$ observations per source. This includes 9,003 new sources that have been added as part of the CSC 2.1 release, in addition to the sources from the previous release.

\section{Methods}\label{sec:methods}

\begin{figure*}
\includegraphics[width=0.99\textwidth]{figures/flowflow.png}
 \caption{Flowchart of the proposed representation learning approach for anomaly detection in time-domain high-energy astrophysics, enabling the systematic detection of transients in high-energy archives. The first step is to create uniform event file representations by binning photon arrival times and energies in the event files into into $E-t$~Maps (Event File Representation I) or $E-t-dt$~Cubes (Event File Representation II). The second step involves extracting informative features from these event file representations via principal component analysis (Feature Extraction I) or sparse autoencoders (Feature Extraction II). The third step is to apply dimensionality reduction to the extracted features and to create a low-dimensional embedding space, which is clustered in the fourth step using density-based clustering. Previously identified transients occupy well-isolated clusters on the edges of the embedding space, thus new transients can be identified by exploring these transient-dominant edge clusters and performing nearest-neighbor searches around known bona-fide flares and dips. Finally, we compile these search results in a publicly available catalog of transient candidates, serving as a fertile ground for the discovery of new X-ray transients.}
 \label{fig:process}
\end{figure*}

In this work, we introduce a novel representation learning based anomaly detection method to systematically search for X-ray transients in high-energy archives. We begin with an overview of the method here and provide detailed explanations of each step in individual subsections. The full pipeline is illustrated in Figure \ref{fig:process}. Starting with the event files described in \S~\ref{sec:data}, we (i) build two novel and uniform event file representations by binning their arrival times and energies into $E-t$~Maps (Event File Representation I) or $E-t-dt$~Cubes (Event File Representation II); (ii) use principal component analysis (Feature Extraction I) or sparse autoencoders (Feature Extraction II) to extract informative features from the event file representations; (iii) apply dimensionality reduction to the extracted features to create a low-dimensional embedding space; (iv) use density-based clustering to create embedding clusters that group event files with similar characteristics, for example transient behavior or certain spectral features. Previously identified transients like the extragalactic magnetar-powered flare candidate reported by \cite{2022ApJ...927..211L} and the extragalactic planet candidate dip reported by \cite{2021NatAs...5.1297D}, shown in Figure \ref{linrosanne}, occupy well-isolated clusters in the embedding space. Exploring these clusters and conducting nearest-neighbor searches enables us to effectively find analogs to bona-fide time-domain anomalies, while at the same time grouping them according to their spectral properties. We compile the identified transient candidates in a catalog. While our approach is designed and tested using \emph{Chandra} data, it is applicable to any dataset consisting of event lists, like those from other high-energy telescopes. The described transient detection approach is applied to both types of event file representations with both feature extraction methods, resulting in four different embeddings. We denote the different cases as described in Table \ref{tab:cases}.

\begin{figure*}
 \centering
 \includegraphics[width=0.495\textwidth]{figures/13814_567.pdf}
 \includegraphics[width=0.495\textwidth]{figures/4062_31.pdf}
\caption{\textit{Left panel:} Lightcurve of the first extragalactic planet candidate dip reported by \protect\cite{2021NatAs...5.1297D} detected in the \emph{Chandra} observation ObsID 13814. \textit{Right panel:} Lightcurve of the magnetar-powered X-ray flare candidate reported by \protect\cite{2022ApJ...927..211L} detected in the \emph{Chandra} observation ObsID 4062.}
 \label{linrosanne}
\end{figure*}

\begin{table}

\centering
\caption{Naming of the different embedding result cases based on the event file representation and feature extraction method.}
\begin{tabular}{lll}
\hline
Case & Event File Representation & Feature Extraction \\
\hline
\it{2D-PCA} & $E-t$~Maps & Principal Component Analysis \\
\it{3D-PCA} & $E-t-dt$~Cubes & Principal Component Analysis \\
\it{2D-AE} & $E-t$~Maps & Sparse Autoencoder \\
\it{3D-AE} & $E-t-dt$~Cubes & Sparse Autoencoder \\
\hline
\end{tabular}
\label{tab:cases}
\end{table}

\begin{figure*}
 \centering
 \includegraphics[width=0.33\textwidth]{figures/nbin_E2.pdf}
 \includegraphics[width=0.33\textwidth]{figures/nbin_t2.pdf}
\includegraphics[width=0.33\textwidth]{figures/nbin_dt2.pdf}
 \caption{The distribution of the optimal number of bins for the energy dimension $n_{\epsilon}$ (left), time dimension $n_{\tau}$ (middle), time difference dimension $n_{\delta\tau}$ (right). The distribution of $n_{\tau}$ only includes event files for which $p_{\rm{var}}^b > 0.9$. The vertical lines indicate the number of bins chosen for the event file representations.}
 \label{Fig:ultiumatebinx}
\end{figure*}

\subsection{Event File Representation} \label{sec:datarepp}
The different event files in the dataset are variable in length $N$ and duration $T$, as shown in Appendix \ref{appendix:distributions}. The large variation in the number of events and duration highlights the challenge in producing uniform data representations that preserve relevant information on time variability and spectral properties. While there exist machine learning architectures that take variable length inputs, the significant differences in the number of events from object to object make standardization of the inputs challenging, even when these architectures are used \citep{2022SPIE12186E..0JM}. As a first step in our analysis, we introduce 2-dimensional and 3-dimensional fixed-length representations based on an informed binning strategy for the event files, similar to the DMDT maps for optical lightcurves introduced by \citet{2017arXiv170906257M}.

\subsubsection{2D Histogram Representation ($E-t$~Maps)}
Assume an event file with $N$ photons and a photon arrival time column $\boldsymbol{t}$ with entries $\{t_k\}_{k=1}^N$ and energy column $\boldsymbol{E}$ with entries $\{E_k\}_{k=1}^N$. The event file duration is given by $T = t_N - t_1$. The energy column entries take values in the broad energy band of \emph{Chandra}'s ACIS instrument, i.e. $E_k \in \left[E_{min}, E_{max}\right]$, where $E_{min}=0.5\,\mathrm{keV}$ and $E_{max}=7\,\mathrm{keV}$ comes from considering appropriate boundaries for the energy response of \emph{Chandra}'s ACIS instrument. Beyond these boundaries, the telescope's aperture effective area is low for the majority of detected sources. First, we obtain the normalized time column, given by $\boldsymbol{\tau} = \frac{\boldsymbol{t} - t_{1}}{T}$, and the logarithm of the energy column, given by $\boldsymbol{\epsilon} = \mathrm{log}\,\boldsymbol{E}$. The resulting boundaries for normalized time column are $\boldsymbol{\tau} \in [\tau_{min}, \tau_{max}]$, where $\tau_{min}=0$ and $\tau_{max}=1$. The range for the log-energy column is $\boldsymbol{\epsilon}\in [\epsilon_{min}, \epsilon_{max}]$, where $\epsilon_{min}=\mathrm{log}\,0.5\,\mathrm{keV}$ and $\epsilon_{max}=\mathrm{log}\,7\,\mathrm{keV}$.

Next, we determine the dimensionality of our representations. For a each event file, we determine the optimal number of bins in the energy dimension, $n_{\epsilon}$, with the Freedman-Diaconis rule \citep{freedman1981histogram}, a widely used method that balances the trade-off between too noisy histograms (too many bins) and not informative enough histograms (too few bins). The optimal bin width $b_{\epsilon}$ according to this rule is calculated in the following way:

\begin{equation}
\quad b_{\epsilon} = 2 \frac{IQR(\boldsymbol{\epsilon})}{N^{~\frac{1}{3}}},
\label{freedman}
\end{equation}
where $IQR(\epsilon)$ represents the interquartile range of the $\epsilon$ values for a given event file of length $N$. Subsequently, we obtain the optimal number of energy bins $n_{\epsilon}$ with:
\begin{equation}
n_{\epsilon} = \frac{\epsilon_{max} - \epsilon_{min}}{b_{\epsilon}}.
\end{equation}

For each event file, we determine the optimal number of bins in the time dimension, $n_{\tau}$, with the help of the Bayesian Blocks algorithm, which was specifically developed for time series analysis in astronomy \citep{bbscargle}. This algorithm partitions the time series into adaptive width bins or blocks that are statistically distinct from neighboring blocks; that is, within a given time-ordered Bayesian block, events grouped in that block are consistent with having a similar event arrival rate. We use the default \astropy implementation of Bayesian blocks, and set the false alarm probability parameter to $p_0 = 0.01$ \citep{astropy}, which implies a 1\% probability of declaring a change of rate when there is none. For each event file, we define the optimal uniform bin width $b_{\tau}$ as the minimum bin width calculated by the Bayesian Blocks algorithm, and then find the optimal number of time bins $n_{\tau}$ with:

\begin{equation}
n_{\tau} = \frac{\tau_{max} - \tau_{min}}{b_{\tau}}.
\label{bb}
\end{equation}

The optimal number of bins is different for each event file, due to their different lengths $N$ and durations $T$. To select a bin size that can be applied to all event files, we consider the distributions of these optimal bin sizes, which are shown in Figure \ref{Fig:ultiumatebinx}. For the distribution of $n_{\tau}$ values we only use those event files for which $p_{\rm{var}}^b > 0.9$. The intent of this is to effectively capture variability timescales that are associated with short time-domain events, such as flares and dips.

We choose the $90$th percentile value of each distribution to set the final number of bins in each dimension. That is, only 10\% of the event files will have an optimal number of bins that is larger than the chosen values $n_{\epsilon}=16$ and $n_{\tau}=24$. The choice of the $90$th percentile, rather than the mean or mode, is motivated by the need to capture sufficient statistical detail even for long event files, while keeping the size of the resulting representations computationally tractable. Choosing a lower resolution would risk losing significant details in the representation, particularly short-duration events such as flares and dips within longer event files. The $E-t$~Maps are the $2$D histogram representations with size $(n_{\tau},n_{\epsilon})=(24,16)$ that result from binning the events according to the optimized number of bins.

Figure \ref{Fig:2drep} shows the $E-t$~Maps for the known extragalactic dip reported by \cite{2021NatAs...5.1297D} and known extragalactic flare reported by \cite{2022ApJ...927..211L}.

\subsubsection{3D Histogram Representation ($E-t-dt$~Cubes)}

We now introduce the $E-t-dt$~Cubes, which extend the $E-t$~Maps by a third dimension that serves as a proxy for the photon arrival rate. For an event file of length $N$, consider the array of time differences between consecutive photon arrivals $\boldsymbol{\Delta t}$ with entries $\Delta t_k = t_{k+1} - t_k$ for $k=1,2,\ldots,N-1$. We again scale and normalize the obtained values, so that they adopt values between $0$ and $1$, using in each case the minimum value $\Delta t_{min}$ and maximum value $\Delta t_{max}$. This provides the third dimension $\boldsymbol{\delta\tau}$:

\begin{equation}
\boldsymbol{\delta\tau} = \frac{\boldsymbol{\Delta t} - \Delta t_{min}}{\Delta t_{max} - \Delta t_{min}}.
\end{equation}

The additional dimension is intended to better isolate short-duration features in time variability by capturing high photon arrival rates, which are typical of flares, as well as very low photon arrival rates, which are typical of dips. The boundaries of our histogram representations in this dimension are $\boldsymbol{\delta\tau} \in [\delta\tau_{min}, \delta\tau_{max}]$, where $\delta\tau_{min}=0$ and $\delta\tau_{max}=1$. We determine the optimal number of bins in the $\boldsymbol{\delta\tau}$ dimension, $n_{\delta\tau}$, again by computing the optimal bin width $b_{\delta\tau}$ with the Freedman-Diaconis rule and dividing the range for $\boldsymbol{\delta\tau}$ by $b_{\delta\tau}$:

\begin{equation}
b_{\delta\tau} = 2 \frac{IQR(\boldsymbol{\delta\tau})}{N^{~\frac{1}{3}}},
\label{freedman2}
\end{equation}

\begin{equation}
n_{\delta\tau} = \frac{\delta\tau_{max} - \delta\tau_{min}}{b_{\delta\tau}}.
\end{equation}

The distribution of $n_{\delta\tau}$ across the event files is shown in Figure \ref{Fig:ultiumatebinx}. Most of the relevant time-domain information is already captured by $\boldsymbol{\tau}$, but adding $\boldsymbol{\delta\tau}$ provides an additional marker for dips and flares that can be shorter than the timescales probed by our chosen binning of $\boldsymbol{\tau}$.

Unlike in the other two dimensions, we choose the $75$th percentile value of the distribution as our final choice of common binning, which results in $n_{\delta\tau}=16$. This is because in order to identify short transients, we need to capture strong deviations in $\boldsymbol{\delta\tau}$ only. Choosing a lower value for $n_{\delta\tau}$ reduces noise an improves computational tractability. Having both $\boldsymbol{\tau}$ and $\boldsymbol{\delta\tau}$ represented also breaks any assumption of stationarity, in that we can be sensitive to transient events happening at any time during the observation of the source, and break degeneracies between periodic and non-periodic features in the representations presented by \cite{2022SPIE12186E..0JM}. The $E-t-dt$~Cubes are the resulting $3$D histogram event file representations with size $(n_{\tau},n_{\epsilon},n_{\delta\tau})=(24,16,16)$.

Figure \ref{Fig:2drep} shows the $E-t-dt$~Cubes for the known extragalactic dip reported by \cite{2021NatAs...5.1297D} and known extragalactic flare reported by \cite{2022ApJ...927..211L}.

\subsubsection{Feature Notation}\label{XYnotation}

The event file representations can now be used as inputs for various statistical learning and machine learning algorithms. For the $i^{th}$ event file in the dataset of length $m =$ 95,473, we denote the corresponding feature vector as $\vec{x}_i = [x_{1}, x_{2}, \ldots, x_{n}]_i$, where $n = n_{\tau} \cdot n_{\epsilon} = 384$ for the $E-t$~Maps and $n = n_{\tau}\cdot n_{\epsilon} \cdot n_{\delta\tau} = 6,144$ for the $E-t-dt$~Cubes. The set of all feature vectors is denoted as $\mathbf{X}=[\vec{x}_1, \vec{x}_2, \ldots, \vec{x}_m]^{\top}$ with size $(m,n)$.

\subsection{Feature Extraction I: Principal Component Analysis}
\label{sec:features}

We use \textit{Principal Component Analysis} (PCA) \citep{pearson1901liii} provided by \texttt{scikit-learn} \citep{scikit-learn} as our first feature extraction method. The extracted principal components should encode relevant time-domain and spectral information of the event file they represent. PCA involves transforming a dataset into a new coordinate system by finding the principal components of the data that capture most of the variance in the data. By projecting the dataset onto principal components, PCA reduces the dimensionality of the data while retaining the most important information, which increases the interpretability of high-dimensional data.

\subsubsection{PCA Algorithm}

We start with the feature vector set $\mathbf{X}$ of size $(m,n)$ representing our dataset with $m$ samples and $n$ dimensions. PCA aims to find a new coordinate system defined by a set of orthogonal axes, i.e. the principal components, that captures the maximum amount of variance in the data. The PCA result is a transformed dataset $\mathbf{X_{pc}}$ obtained by projecting $\mathbf{X}$ onto the principal components:

\begin{equation}
\mathbf{X_{pc}} = \mathbf{X}\mathbf{W},
\end{equation}
where $\mathbf{W}$ is matrix of size $(n,n_{pc})$ containing the first $n_{pc}$ principal components to be retained as its columns and $\mathbf{X_{pc}}$ is of size ($m$, $n_{pc}$) with a reduced dimensionality of $n_{pc}$. For a more detailed explanation of the algorithm, we refer the reader to \cite{jolliffe2002principal}.

\subsubsection{Principal Components Retained}

The main PCA hyperparameter is the number of principal components $n_{pc}$ to retain. Figure \ref{Fig:scree} shows two scree plots illustrating the amount of variance explained by each principal component in descending order and the cumulative proportion of variance explained by the principal components for both $E-t$~Maps and $E-t-dt$~Cubes. A common approach to determine the optimal value of $n_{pc}$ is to find the knee point in the cumulative scree plot of the principal components. This balances the objective of minimizing the dimensionality while retaining as much information as possible. Defining the knee point as the point beyond which adding additional principal components increases the amount of variance by less than $0.1$\% gives $n_{pc}=15$ for $E-t$~Maps and $n_{pc}=22$ for $E-t-dt$~Cubes as indicated in Figure \ref{Fig:scree}. These capture $94.1$\% and $89.9$\% of the variance respectively.
\begin{figure}
 \centering
 \includegraphics[width=0.49\textwidth]{figures/pca2dscree.pdf}
 \includegraphics[width=0.49\textwidth]{figures/pca3dscree.pdf}
 \caption{Scree plot for the principal components of the $E-t$~Maps (top) and $E-t-dt$~Cubes (bottom). The scree plots show the amount of variance explained by each individual principal component including the knee point in the cumulative variance.}
 \label{Fig:scree}
\end{figure}

\subsection{Feature Extraction II: Sparse Autoencoder Neural Network}

As an alternative to PCA, we now build \emph{Autoencoder} \citep{hinton2006reducing} models with \texttt{TensorFlow} \citep{tensorflow2015-whitepaper} to learn a set of latent features from the $E-t$~Maps and $E-t-dt$~Cubes that can be used to isolate transients and encode specific spectral properties. An autoencoder is composed of two neural networks, an encoder and a decoder, which work together to learn a compressed representation of the input data. The encoder network takes the input data and maps it to a lower-dimensional representation, often called `latent space' or `bottleneck'. The number of neurons in the bottleneck determines the dimensionality of the learned representation. The decoder network then aims to reconstruct the original input from this compressed representation. The decoder is typically a mirrored version of the encoder gradually upsampling the latent space until the output matches the dimensions of the original input. By minimizing the reconstruction error between input and output during training, the model learns a low-dimensional representation of the input. The bottleneck forces the encoder to capture the most important information necessary for accurate reconstruction, effectively compressing the input and learning to extract informative features in an unsupervised manner. Once the autoencoder is trained, the encoder network can be used as a standalone feature extractor to obtain a compressed representation of the input data, which can be used for downstream tasks such as clustering or anomaly detection. As opposed to PCA, which is a linear technique that works well for linearly correlated data but fails to capture complex non-linear relationships, an autoencoder is able to learn complex non-linear relationships. We design two different autoencoders to process the $E-t$~Maps and $E-t-dt$~Cubes.

\begin{figure}
 \centering
 \includegraphics[width=0.5\textwidth]{figures/ae_new1.png}
 \includegraphics[width=0.5\textwidth]{figures/ae_new2.png}
 \caption{Encoder architecture of the convolutional autoencoder used for the $E-t$~Maps (top) and of the fully connected autoencoder used for the $E-t-dt$~Cubes (bottom). The decoder architecture is simply a mirror image of the encoder.}
 \label{xaecnn}
\end{figure}

\subsubsection{Convolutional Autoencoder}

In a convolutional autoencoder \citep{masci2011stacked}, both the encoder and decoder network consist of convolutional layers \citep{lecun1998gradient}, which perform convolutions over the input using a filter. These filters are small matrix kernels with learnable weights that slide across the input, allowing the network to capture high-level features while preserving important spatial hierarchies and relationships, which is why they are often used for image-like data. This makes this architecture particularly well-suited to recognize spatial patterns such as dips or flares in our $E-t$~Maps. To gradually reduce the dimension of the input while it is being passed through the encoder network, we use stride convolution layers \citep{simonyan2014very} with a stride value of 2 for downsampling. This means that the learnable filter jumps two pixels at a time as it slides over the input. The output of the convolutional layers is a feature map, which is then flattened to a feature vector and passed through a series of fully connected layers, where every neuron in the previous layer is connected to every neuron in the next layer. These fully connected layers are responsible for mapping the learned features to a lower-dimensional latent representation in the bottleneck and perform non-linear transformations while downsampling through the use of non-linear activation functions. The final latent space has $n_{ae}=12$ elements, representing the most essential features of the input data, which can now be used for further downstream tasks. Figure \ref{xaecnn} shows a diagram of the encoder part of the model and Table \ref{tab:autoencoder} summarizes its architecture.

\begin{table}
\centering
\caption{Summary of the encoder architecture of the convolutional autoencoder used to extract informative features from the $E-t$\,Maps. Note that each layer has a Leaky ReLU activation function and that each standard convolutional layer is followed by batch normalization with momentum 0.9.}
\begin{tabular}{|l|l|l|l|l}
\hline
Layer & Output Shape & Filters & Kernel & Stride\\ \hline
Input & (24, 16) & - & - & - \\
Convolution & (24, 16) & 32 & (3, 3) & - \\
Convolution & (12, 8) & 32 & (2, 2) & 2 \\
Convolution & (12, 8) & 16 &(3, 3) & - \\
Convolution & (6, 4) & 16 & (2, 2) & 2 \\
Flatten & 384 & - & - & - \\
Dense & 192 & - & - & - \\
Dense & 48 & - & - & - \\
Dense (Bottleneck) & 12 & - & - & - \\
\hline
\end{tabular}
\label{tab:autoencoder}
\end{table}

\subsubsection{Fully Connected Autoencoder}

Our $E-t-dt$~Cubes introduce an additional dimension resulting in sparse $3$D input data. Convolutional layers assume regular grid-like data, making them less effective for handling sparse data. Moreover, very expensive $3$D convolutional operations would substantially increase complexity of the model. Therefore, we use a simple fully connected autoencoder for the $E-t-dt$~Cubes. Its encoder network consists of a series of fully connected layers, which gradually map the original input data to a latent space with $n_{ae} = 24$ elements. Figure \ref{xaecnn} shows a diagram of the encoder part of the model and Table \ref{tab:autoencoder2} summarizes its architecture.

\begin{table}
\centering
\caption{Summary of the encoder architecture of the fully connected autoencoder used to extract informative features from the $E-t-dt$\,Cubes. Note that each layer has a Leaky ReLU activation function and that each standard fully-connected layer is followed by batch normalization with momentum 0.9.}
\begin{tabular}{|l|l|}
\hline
Layer & Output Shape \\ \hline
Input & (24, 16, 16) \\
Flatten & 6144 \\
Dense & 1536 \\
Dense & 384 \\
Dense & 92 \\
Dense (Bottleneck) & 24 \\
\hline
\end{tabular}
\label{tab:autoencoder2}
\end{table}
\subsubsection{Activation Functions}

Neural networks are able to learn and represent complex non-linear relationships due to the introduction of non-linear activation functions within their layers. An activation function is a mathematical function used in a neural network to determine whether a neuron should be activated or not, based on its input. It essentially decides how much of the input signal should pass through the neuron, producing an output that can either be passed to the next layer or used to make predictions. The popular Rectified Linear Unit (ReLU) activation function $ReLU(x) = \text{max}(0,x)$ \citep{nair2010rectified} is simple and computationally efficient. To mitigate any potential encounters of the `dying the ReLU problem', where neurons become non-responsive during training, we choose an extended version called Leaky ReLU \citep{maas2013rectifier}:
\begin{equation}
LeakyReLU(x) = \text{max}(\alpha x, x),
\end{equation}
where $\alpha=0.1$ is a hyperparameter that defines the slope of the function for negative input values. ReLU sets all negative values in the input to zero, while Leaky ReLU allows a small negative slope for negative inputs, which can help prevent neurons from dying. As for the output layer, we want any values to be mapped to a range between $0$ and $1$, which is achieved by using the sigmoid activation function:
\begin{equation}
sigmoid(x) = \frac{1}{1 + e^{-x}}.
\end{equation}

\subsubsection{Loss Function and Sparsity Regularization}

In order to encourage the autoencoder to generate reconstructions close to the original inputs, we use the mean squared error ($MSE$) as as a measure of the reconstruction quality given by:

\begin{equation}
MSE = \frac{1}{m} \sum_{i=1}^{m} (x_i - \hat{x}_i)^2,
\end{equation}
where $x_i$ is the $i^{th}$ element of the input vector and $\hat{x}_i$ is the corresponding is reconstructed output. The $MSE$ is a straightforward measure of reconstruction error, and its differentiability allows efficient gradient computation for updating model weights via gradient-based optimization.

Our neural networks are so called sparse autoencoders \citep{ng2011sparse}, which promote sparsity in the learned representation, meaning only a small subset of the neurons in the network are active at any given time. Sparse representations are valuable for our work because they help extract highly informative features from the input, while disregarding irrelevant or noisy information. To encourage sparsity in the latent space, we introduce a L1 regularization term in the objective, resulting in the following loss function:
\begin{equation}
L = MSE + \lambda \cdot \sum_{j=1}^{n_w} \lvert w_j \rvert = \frac{1}{m} \sum_{i=1}^{m} (x_i - \hat{x}_i)^2 + \lambda \cdot \sum_{j=1}^{n_w} \lvert w_j \rvert,
\end{equation}
where $\lambda=0.1$ is the regularization strength and $w_j$ are the individual bottleneck weight values of which there are $n_w$ in total. L1 regularization pushes small weights to zero and thus helps the model prioritize the most significant features of the input data, leading to a semantically meaningful latent space.

\subsubsection{Training}

Starting with the original dataset with a $m =$ 95,473 samples and using a test split of $0.1$ gives us a training and validation set of length 85,925 and a test set of length 9,548. Further using a validation split of $0.2$, gives 68,740 samples for training and 17,185 for validation. We run the training process for a maximum of 200 epochs with a batch size of 1,024 samples. The initial learning rate was set to $0.01$ along with an on plateau learning rate scheduler, which dynamically reduces the learning rate by a factor of $0.1$ if the validation loss plateaus for longer than $10$ epochs. Reducing the learning rate when a plateau is detected can help escape local minima in the loss surface and converge to a more optimal solution in the parameter space. This scheduler is used in combination with the Adaptive Moment Estimation (Adam) optimizer \citep{kingma2014adam}, which is a stochastic gradient descent algorithm combining the benefits of both adaptive learning rates \citep{duchi2011adaptive} and momentum-based optimization techniques \citep{sutskever2013importance}. Finally, we use an early stopping callback to monitor the validation loss. It automatically interrupts the training process if the validation loss does not improve for $25$ epochs and restores the weights of the model to the best observed weights during training. The training process for both autoencoder models is shown in Appendix \ref{appendix:training}. Once the autoencoder is trained, we can use the encoder to transform the original dataset $\mathbf{X}$ to the feature vector space $\mathbf{X_{ae}}$ of size ($m$, $n_{ae}$) with a reduced dimensionality of $n_{ae}$ features.

\subsection{Dimensionality Reduction}

Using \textit{t-SNE} \citep{maaten2008visualizing}, short for t-Distributed Stochastic Neighbor Embedding, we create two-dimensional embeddings of the informative features previously extracted from the event file representations using PCA or sparse autoencoders. The t-SNE algorithm is a method used to map the input data onto a low-dimensional embedding space, and is particularly useful for the visualization of clusters and patterns in high-dimensional datasets. Each high-dimensional sample is transformed into a low-dimensional embedding in such a way that similar object are nearby points, while dissimilar objects are distant points in the embedding space. Essentially, it aims to capture the local structure of the data by preserving the pairwise similarities between objects while mapping them to a lower-dimensional embedding space.

\subsubsection{Algorithm}

We use our informative features, $\mathbf{X_{if}}=\mathbf{X_{pc}}$ or $\mathbf{X_{if}}=\mathbf{X_{ae}}$, as input to the t-SNE algorithm to reduce the data to a two-dimensional embedding, denoted as $\mathbf{Z}$. First, t-SNE creates a probability distribution $P$ for pairs of high-dimensional data points in $\mathbf{X_{if}}$, assigning higher probabilities to similar pairs and lower probabilities to dissimilar ones. This is done by modeling pairwise similarities using a Gaussian kernel with a specific perplexity parameter, which controls the effective number of neighbors considered for each point. Next, t-SNE defines a similar probability distribution $Q$ for the pairwise similarities in the low-dimensional space $\mathbf{Z}$, modeled using a Student's t-distribution. The goal of t-SNE is to minimize the difference between $P$ and $Q$ using gradient descent, with the Kullback-Leibler (KL) divergence \citep{kullback1951information} as the cost function:
\begin{equation} D_{KL}(P\,|\,Q) = \sum_{i \neq j} P_{ij} \log\frac{P_{ij}}{Q_{ij}},
\end{equation} where $P_{ij}$ and $Q_{ij}$ represent pairwise similarities in the high- and low-dimensional spaces, respectively. The algorithm iteratively adjusts the low-dimensional embedding $\mathbf{Z}$ to minimize the KL divergence, often requiring hundreds to thousands of iterations for convergence. The result of this optimization is a two-dimensional representation $\mathbf{Z}$ of size $(m, 2)$, where similar points in the high-dimensional space are clustered closely together.
\subsubsection{Hyperparameter Optimization}

The t-SNE algorithm has a number of important hyperparameters to be tuned. The two most important parameters are the \texttt{perplexity} and the \texttt{learning\_rate}. The \texttt{perplexity} parameter controls the balance between capturing the local versus global structure in the data, while the \texttt{learning\_rate} controls the step size at each iteration of the optimization process. The \texttt{n\_iter} parameter is the number of iterations. To ensure reproducibility, we set a fixed \texttt{random\_state}. Our t-SNE hyperparameter optimization approach is detailed in Appendix \ref{appendix:hyperparameters}. A summary of the final t-SNE hyperparameters is provided in Table \ref{tab:tsne1}.

\begin{table}

\centering
\caption{Chosen t-SNE hyperparameters for different embedding cases.}
\begin{tabular}{lllll}
\hline
Hyperparameter & \it{2D-PCA} & \it{3D-PCA} & \it{2D-AE} & \it{3D-AE}\\
\hline
\texttt{perplexity} & 30 & 50 & 40 & 60 \\
\texttt{learning\_rate} & 80 & 120 & 100 & 180 \\
\texttt{n\_{iter}} & 4500 & 3500 & 3000 & 2000\\
\texttt{random\_state} & 11 & 11 & 2412 & 12\\
\hline
\end{tabular}
\label{tab:tsne1}
\end{table}

\subsection{Clustering}

The next step is the identification of individual clusters in the embedding space using \dbscan \citep{hartigan1979algorithm}, short for Density-Based Spatial Clustering of Applications with Noise. Unlike traditional clustering algorithms such as k-means, DBSCAN does not require the number of clusters to be specified, as it identifies dense regions in the data space based on a density criterion.

\subsubsection{Algorithm}

We use our t-SNE embedding space $\mathbf{Z}$ as input to the DBSCAN algorithm, which segments the embedding space into multiple clusters. The DBSCAN algorithm has two main hyperparameters. The \texttt{eps} parameter defines the radius of the neighborhood surrounding each point in the dataset, while the \texttt{minPts} parameter specifies the minimum number of points required within this neighborhood for a data point to be classified as a core point. A border point is defined as a point that is in the vicinity of at least one core point but has fewer than \texttt{minPts} within its neighborhood. All other points are considered to be noise points. Clusters are then created from the aggregation of core points and their associated border points, with noise points being categorized as outliers. Figure \ref{dbscan} visualizes the clustering method.

\subsubsection{Hyperparameter Optimization}

Our DBSCAN hyperparameter optimization approach is detailed in Appendix \ref{appendix:hyperparameters}. A summary of the final t-SNE hyperparameters is provided in Table \ref{tab:dbscan1}.

 \begin{figure}
 \centering
\includegraphics[width=0.5\columnwidth]{figures/dbscan_illustration.png}\vspace{10pt}
 \caption{Illustration of the DBSCAN clustering algorithm, showing core points as densely connected regions, border points along cluster edges, and noise points as outliers. Adapted from \protect\cite{2024Icar..41915777S} with permission.}
 \label{dbscan}
\end{figure}

\begin{table}

\centering
\caption{Chosen DBSCAN hyperparameters for different embedding cases.}
\begin{tabular}{lllll}
\hline
Hyperparameter & \it{2D-PCA} & \it{3D-PCA} & \it{2D-AE} & \it{3D-AE}\\
\hline
\texttt{eps} & 2.8 & 2.1 & 1.6 & 1.9\\
\texttt{minPts} & 26 & 23 & 25 & 24 \\
\hline
\end{tabular}
\label{tab:dbscan1}
\end{table}

\subsection{Previously Reported Transients}

We highlight the embeddings of previously reported bona-fide transients, listed in Table \ref{tab:bonafide_flares}, in our low-dimensional representation space to identify transient-dominant clusters. The flares include extragalactic FXTs reported by \cite{2013ApJ...779...14J}, \cite{2015MNRAS.450.3765G}, \cite{2019MNRAS.487.4721Y},
\cite{2021ATel14599....1L},
\cite{2022ApJ...927..211L}, \cite{quirola2022extragalactic} and a set of stellar flares found in the dataset by manual inspection. The dips include the extragalactic planet candidate in M\,51 reported by \cite{2021NatAs...5.1297D}, the ultraluminous X-ray source (ULX) $2$E $1402.4$+$5440$ in NGC\,5457 \citep{2002ApJS..143...25C,2004ApJS..154..519S} and the well-studied eclipsing low-mass X-ray binary (LMXB) EXO 0748$-$676 \citep{1986ApJ...308..199P, 2014A&A...564A..62D}. These transients occupy well-isolated clusters. Exploring transient-dominant clusters and performing nearest-neighbor searches around known transients allows us to find new transients.

 \begin{table*}

\centering
\caption{Previously reported flares and dips used to identify transient-dominant clusters.}
\begin{tabular}{lllll}
\hline
CSC Name & ObsID & Date & Transient Type & Description\\
\hline
 2CXO J123605.1+622013 & 957 & 2000-02-23 & Flare & Extragalactic Fast X-ray Transient \citep{2019MNRAS.487.4721Y}\\
 2CXO J122531.5+130357 & 803 & 2000-05-19 & Flare& Extragalactic Fast X-ray Transient \citep{2013ApJ...779...14J}\\
 2CXO J190725.1+070906 & 1042 & 2001-09-06 & Flare & Unknown Origin\\
 2CXO J111128.3+554021 & 2025 & 2001-09-08 & Flare & Extragalactic Fast X-ray Transient \citep{quirola2022extragalactic} \\
 2CXO J123625.3+621405 & 3389 & 2001-11-21 & Flare& Extragalactic Fast X-ray Transient \citep{2019MNRAS.487.4721Y}\\
 2CXO J111908.8-612540 & 2833 & 2002-03-31 & Flare & X-ray Source \\
 2CXO J053517.5-051739 & 4395 & 2003-01-08 & Flare & Stellar Flare (Orion Variable)\\
 2CXO J053528.1-051856 & 4396 & 2003-01-18 & Flare & Stellar Flare (Young Stellar Object)\\
 2CXO J163553.8-472540 & 3877 & 2003-03-24 & Flare & Compact Object System \citep{2012ApJ...756...27L}\\
 2CXO J050706.7-315211 & 4062 & 2003-05-10 & Flare & Extragalactic Fast X-ray Transient \citep{2022ApJ...927..211L}\\
 2CXO J151457.6+364817 & 3988 & 2003-10-05 & Flare & Stellar Flare\\
 2CXO J165334.4-414423 & 6291 & 2005-07-16 & Flare & Magnetic Cataclysmic Variable \citep{2014ApJ...780...39L}\\
 2CXO J025616.7+585756 & 7151 & 2006-06-21 & Flare & Unknown Origin\\
 2CXO J074111.5+741450 & 10822 & 2009-06-18 & Flare & Unknown Origin\\
 2CXO J112017.5+125818 & 9548 & 2008-03-31 & Flare & Extragalactic Fast X-ray Transient \citep{quirola2022extragalactic} \\
 2CXO J140828.9-270328 & 12884 & 2011-01-03 & Flare & Extragalactic Fast X-ray Transient \citep{2015MNRAS.450.3765G}\\
 2CXO J010344.5-214845 & 13454 & 2011-09-19 & Flare & Extragalactic Fast X-ray Transient \citep{2022ApJ...927..211L}\\
 2CXO J064114.4+093321 & 13610 & 2011-12-05 & Flare & Stellar Flare (RS CVn Variable)\\
 2CXO J064028.7+093059 & 14368 & 2011-12-03 & Flare & Stellar Flare (T Tauri Star)\\
 2CXO J064119.6+093144 & 14368 & 2011-12-03 & Flare & Stellar Flare (T Tauri Star) \\
 2CXO J095959.4+024646 & 15211 & 2012-12-13 & Flare & Extragalactic Fast X-ray Transient \citep{2019MNRAS.487.4721Y}\\
 2CXO J235212.2-464343 & 13506 & 2012-08-30 & Flare & Extragalactic Fast X-ray Transient \citep{2015MNRAS.450.3765G}\\
 2CXO J030309.0-774435 & 15113 & 2014-03-27 & Flare & Extragalactic Fast X-ray Transient \citep{quirola2022extragalactic} \\
 2CXO J234503.4-423841 & 20635 & 2017-08-31 & Flare & Extragalactic Fast X-ray Transient \citep{2022ApJ...927..211L} \\
 2CXO J134856.4+263944 & 24604 & 2021-04-23 & Flare & Extragalactic Fast X-ray Transient \citep{2021ATel14599....1L} \\
 \hline
 2CXO J121656.9+374335 & 942 & 2000-05-20 & Dip & Ultra-luminous X-ray Source\\
 2CXO J140414.2+542604 & 4733 & 2004-05-07 & Dip & Ultraluminous X-ray Source (2E 1402.4+5440)\\
 2CXO J140414.2+542604 & 5322 & 2004-05-03 & Dip & Ultraluminous X-ray Source (2E 1402.4+5440)\\
 2CXO J140414.2+542604 & 4736 & 2004-11-01 & Dip & Ultraluminous X-ray Source (2E 1402.4+5440)\\
 2CXO J140515.6+542458 & 6152 & 2004-11-07 & Dip & Active Galactic Nucleus\\
 2CXO J140414.2+542604 & 6170 & 2004-12-22 & Dip & Ultraluminous X-ray Source (2E 1402.4+5440)\\
 2CXO J140414.2+542604 & 4737 & 2005-01-01 & Dip & Ultraluminous X-ray Source (2E 1402.4+5440)\\
 2CXO J074833.7-674507 & 9070 & 2008-10-12 & Dip & Low-Mass X-ray Binary (EXO 0748$-$676) \\
 2CXO J074833.7-674507 & 10783 & 2008-10-15 & Dip & Low-Mass X-ray Binary (EXO 0748$-$676) \\
 2CXO J021404.0+275239 & 9550 & 2008-10-03 & Dip & Ultra-luminous X-ray Source Candidate\\
 2CXO J074833.7-674507 & 10871 & 2009-02-25 & Dip & Low-Mass X-ray Binary (EXO 0748$-$676) \\
 2CXO J074833.7-674507 & 11059 & 2010-04-20 & Dip & Low-Mass X-ray Binary (EXO 0748$-$676) \\
 2CXO J031702.5-410714 & 11272 & 2010-05-04 & Dip & Ultraluminous X-ray Source (NGC 1291 PSX-2)\\
 2CXO J132939.4+471243 & 13813 & 2012-09-09 & Dip & HII Region\\
 2CXO J132939.9+471236 & 13812 & 2012-09-12 & Dip & HII Region \citep{2004ApJ...609..710D}\\
 2CXO J132943.3+471134 & 13814 & 2012-09-20 & Dip & Extragalactic Planet Transit Candidate \citep{2021NatAs...5.1297D}\\
 2CXO J132939.9+471236 & 13814 & 2012-09-20 & Dip & HII Region \citep{2004ApJ...609..710D}\\

\hline
\end{tabular}
\label{tab:bonafide_flares}
\end{table*}

\subsection{Candidate Selection}

New transients are identified in embedding clusters containing previously reported transients. For well-isolated clusters containing known discovered transients, we use the entire cluster to define new transient candidates. The well-isolated transient-dominant clusters used for candidate selection are listed in Appendix \ref{appendix:transdomclus}. However, in a few cases known transients reside within larger poorly separated clusters. Selecting the entire cluster would result in a high number of false positives. To address this, we instead use the k-nearest neighbors (\emph{knn}) algorithm \citep{cover1967nearest}, identifying the 50 nearest neighbors for each known transient residing in a poorly separated cluster to define additional transient candidates.

\subsection{Cross Matching}

We use an existing cross-match table \citep{cross_match_table} between CSC 2.1 and five other catalogs - Gaia DR3 \citep{2021A&A...649A...1G}, DESI Legacy Survey DR10 \citep{2019AJ....157..168D}, PanSTARRS-1 \citep{2016arXiv161205560C}, 2MASS \citep{2006AJ....131.1163S}, and the SDSS DR$17$ catalog - to complement the X-ray properties with multi-wavelength observations. This includes catalog identifiers, positions, magnitudes, source type classifications and other columns. We cross-matched our transient candidates with the SIMBAD database \citep{2000A&AS..143....9W} by associating each candidate with the nearest SIMBAD object, provided the object is located within a 5\,arcsec radius of the candidate's coordinates listed in the CSC. The multi-wavelength observations of the transient candidates provide valuable information for their characterization and classification.

\begin{table*}
\centering
\caption{The first 5 samples of our transient candidates catalog showing a subset of selected columns.}
\label{tab:cata10}
\begin{tabular}{lllllllll}
\hline
\texttt{CATALOG\_{ID}} & \texttt{CSC\_{name}} & \texttt{TRANSIENT\_{TYPE}} & \texttt{CSC\_{ra}} & \texttt{CSC\_{dec}} & ... &
\texttt{CSC\_var\_index\_b} & ... & \texttt{SIMBAD\_{otype}} \\
\hline
10049\_3 & 2CXO J162636.5-515630 & Dip & 246.652136	& -51.941847 & ... & 7.0 & ... & HighMassXBin \\
10059\_961	& 2CXO J174805.3-244656	& Flare	& 267.022413	& -24.782385 & ... & 8.0 & ... & X \\
10059\_967	& 2CXO J174805.2-244647	& Flare &	267.021774	& -24.779915 & ... & 8.0 & ... & LowMassXBin \\
10062\_450	& 2CXO J152010.7-571110	 & Flare & 	230.044590	& -57.186057 & ... & 10.0 & ... & Unknown \\
10065\_31	& 2CXO J170029.9-461310	& Flare & 255.124760 &	-46.219472 & ... & 9.0 & ... & Unknown \\
\hline
\end{tabular}
\end{table*}

 \begin{figure*}
 \centering
\includegraphics[width=0.3275\textwidth]{figures/16527_79.pdf}
\includegraphics[width=0.3275\textwidth]{figures/2017_59.pdf}
\includegraphics[width=0.3275\textwidth]{figures/5197_13.pdf}

\includegraphics[width=0.3275\textwidth]{figures/1967_69.pdf}
\includegraphics[width=0.3275\textwidth]{figures/7460_88.pdf}
\includegraphics[width=0.3275\textwidth]
{figures/1642_19.pdf}

\includegraphics[width=0.3275\textwidth]{figures/18671_1182.pdf}
\includegraphics[width=0.3275\textwidth]{figures/2227_220.pdf}
\includegraphics[width=0.3275\textwidth]{figures/11721_57.pdf}

\includegraphics[width=0.3275\textwidth]
{figures/4732_39.pdf}
\includegraphics[width=0.3275\textwidth]{figures/19037_17.pdf}
\includegraphics[width=0.3275\textwidth]
{figures/3482_166.pdf}

\includegraphics[width=0.3275\textwidth]{figures/12823_136.pdf}
\includegraphics[width=0.3275\textwidth]
{figures/3877_49.pdf}
\includegraphics[width=0.3275\textwidth]{figures/18706_2047.pdf}

\caption{Lightcurves in the 0.5--7\,kev energy range for different examples of dips (blue), flares (red) and pulsating or quasi-periodic sources (green) in the transient candidates catalog. The shown pulsating or quasi-periodic sources are part of the flare candidates.}
\label{Fig:dip_ex}
\end{figure*}

\section{Results and Discussion} \label{sec:results}
We now present the results of applying the methods in \S~\ref{sec:methods} to the set of representations of X-ray event files in the dataset from \S~\ref{sec:data}.

\subsection{Representation Embedding Space and Clusters}

Figure \ref{Fig:tsneHAR} shows the t-SNE embedding space for the \textit{3D-PCA} and \textit{3D-AE} cases color-coded by the hardness ratio $HR_{\rm{hs}}$. The embedding space for the other two cases, \textit{2D-PCA} and \textit{2D-AE}, are shown in Appendix \ref{appendix:embeddings}. The observed hardness ratio gradients in all embedding spaces indicate that the learned representations effectively encode spectral information, in particular at the level of individual clusters, allowing for the identification of X-ray sources with specific spectral signatures. For the \textit{2D-PCA} and \textit{2D-AE} cases, these gradients are more uniform across the embedding space, because the temporal and spectral information of event files are captured by one axis each in the $E-t$\,Maps. Moreover, some clusters consist exclusively of soft or hard sources, demonstrating that our representations can be leveraged not only to identify transients but also to find analogs to sources with specific spectral characteristics.

Figure \ref{Fig:tsneVAR} shows the \textit{3D-PCA} and \textit{3D-AE} embedding spaces, now color-coded by the variability index $I_{\rm{index}}^b$ with the other two cases shown in Appendix \ref{appendix:embeddings}. The learned embeddings also encode the temporal behavior of the sources, with some clusters being dominated by X-ray detections with significant variability, including transient behavior. To demonstrate this, we also highlight the embeddings of the bona-fide flares and dips listed in Table \ref{tab:bonafide_flares}. Note that these occupy very well-defined clusters on the edges of the representation space, allowing for queries of analog transient behavior. In the \textit{2D-PCA} and \textit{2D-AE} cases, transient sources are distributed across multiple small clusters on the edges of the embedding spaces. In contrast, the \textit{3D-PCA} and \textit{3D-AE} embedding spaces achieve a significantly more compact clustering of bona-fide transients because temporal features in the event files are given a higher importance by the introduction of an additional time-related axis in the $E-t-dt$\,Cubes.

Figure \ref{fig:dbscanClusters} shows the clusters identified by the DBSCAN algorithm in the \textit{3D-PCA} and \textit{3D-AE} cases. The clusters for the other two cases, \textit{2D-PCA} and \textit{2D-AE}, are shown in Appendix \ref{appendix:embeddings}. The largest cluster in all cases (Cluster 1) corresponds to observations that are not 'anomalous', for example non-variable sources or noisy detections in the low-count regime. We also see multiple smaller clusters on the edges of the embedding space clearly separated from this main cluster. Of special interest are clusters that contain known discovered transients, as these likely host other interesting transients that have not yet been discovered. Some of the edge clusters group observations with similar temporal and spectral behavior. For example, Cluster 4 in the \textit{3D-PCA} case only contains flares with high hardness ratios. Other clusters instead group observations primarily by similar temporal behavior, but then show a within-cluster grouping of similar spectral behaviors. For example, Cluster 4 in the \textit{3D-AE} case contains many dipping sources, but show a hardness ratio gradient within the cluster. When comparing the results of different feature extraction methods, we observe that in the \textit{3D-AE} embedding space, nearly all previously identified extragalactic FXTs live within a single, well-isolated cluster (Cluster 8). In contrast, the \textit{3D-PCA} embedding space distributes these extragalactic FXTs across multiple clusters. All of these points underline the effectiveness of our method and that the created representation space is highly informative.

\subsection{Catalog of X-ray Flare and Dip Candidates}

We identify new transient candidates within clusters that are occupied by previously reported transients and by conducting nearest-neighbor searches around these known transients. We compile these in a catalog of X-ray transient candidates, which includes both flares and dips. The selected clusters used to define the new flare and dip candidates in addition to the 50 nearest neighbors of each bona-fide transient are given in Appendix \ref{appendix:transdomclus}. Note that from each selected flare cluster, we include only X-ray detections with a variability index $I_{\rm{var}}^b\geq5$, corresponding to at least 90\% confidence in variability per the Gregory-Loredo algorithm, ensuring statistical significance. We also include a select group of interesting sources identified as non-clustered points within the embeddings, particularly pulsating or quasi-periodic sources, to the flare candidates. Lastly, we manually exclude a fraction of false positives identified by visual inspection of the lightcurves. The resulting catalog contains a total of 3,559 detections (3,447 flares and 112 dips), with the catalog columns described in Appendix \ref{appendix:columns}. Table \ref{tab:cata10} shows the first 5 samples in our catalog for a subset of columns. Figure \ref{Fig:dip_ex} shows a number of example lightcurves of the dips and flares in our catalog. The dip selection shows dips from LMXBs, a high-mass X-ray binary (HMXB), the glitching pulsar PSR\,J0537$-$6910 \citep{2004ApJ...603..682M}, a cataclysmic binary, and the lensed quasar HE\,0230$-$2130 \citep{1999A&A...348L..41W}. The flare selection shows flares from an eruptive variable, a RS CVn variable, a brown dwarf candidate, a HMXB, the redback pulsar PSR\,J1048+2339 \citep{2016ApJ...823..105D} and an active galacti nucleus (AGN). We also show pulsating or quasi-periodic lightcurves from a magnetic cataclysmic variable, the peculiar LMXB IGR\,J16358$-$4726 \citep{2003IAUC.8109....2K, 2004ApJ...602L..45P, 2007ApJ...657..994P} and a pulsar. Figure \ref{Fig:dip_distirbutions} shows the distribution of SIMBAD object types in our transient catalog. About 25\% of the transient candidates do not have a SIMBAD match, making them particularly interesting sources for new transient discoveries. Our dip candidates include 6 \emph{Chandra} observations with prominent dips from the known source CXOGlb J002400.9$-$720453 in the globular cluster NGC\,104 (47\,Tuc). The catalog identifiers for these are \texttt{CATALOG\_ID: 2737\_139, 16527\_79, 15747\_79, 16529\_79, 15748\_79, 16528\_14}. Our flare candidates include a newly discovered extragalactic FXT, which is characterized and discussed in detail in \S~\ref{sec:xrt200515}. Its catalog identifier is \texttt{CATALOG\_ID: 23022\_122}. We recommend using our catalog to identify a diverse range of flares and dips. While this work is primarily motivated by the discovery of new extragalactic transients, we intentionally did not exclude galactic stellar flares to enable systematic follow-up studies to study flare incidence rates, the rotational evolution of stars and more. Users interested exclusively in extragalactic transients can filter out galactic sources using metadata from the CSC and the cross-match columns in the catalog.

\subsection{XRT\,200515: A New Extragalactic Fast X-ray Transient} \label{sec:xrt200515}

Among the flare candidates in our catalog, we discovered an intriguing new extragalactic \emph{Chandra} FXT in an observation of the supernova remnant SNR 0509$-$67.5 in the LMC on May 15, 2020 \citep{2022AJ....164..231G}. What made this transient stand out from thousands of other flares discovered in this work is the unique temporal variability in its lightcurve, which exhibits no detectable pre-flare X-ray emission, a sharp rise of at least 4 orders of magnitude in the count rate to peak intensity followed by a sharp fall, all in a matter of a $<$10\,s, down to $\sim$800\,s long oscillating tail. There is also notable spectral variability during the flare, characterized by an initially hard spectrum at the peak, followed by spectral softening in the tail. The combination of these temporal and spectral properties establishes this transient as the first of its kind within the sample of discovered \emph{Chandra} FXTs. We designate this newly discovered FXT as XRT\,200515 and present a detailed study and discussion of its potential origins.

\subsubsection{X-Ray Detection by Chandra}

The transient XRT\,200515 was detected in \emph{Chandra} ObsID 23022. The target of the observation was the supernova remnant SNR 0509$-$67.5 in the LMC, which is shown in Figure \ref{Fig:xrt_in_sky} alongside the newly discovered FXT event. Table \ref{tab:nstable} summarizes the properties of XRT\,200515 and its associated \emph{Chandra} source 2CXO J051117.2$-$672556 in ObsID 23022. The transient was captured by the ACIS camera in the S4 chip, and is located significantly off-axis in this observation, at an angular distance of 11.75\,arcmin from the aimpoint in the S3 chip. This leads to an elongated and relatively large PSF, which, in this case, is advantageous as it substantially reduces photon pile-up in the initial spike, by spreading the counts over many pixels. We processed the data of \emph{Chandra} observation ObsID 23022 with the Chandra Interactive Analysis of Observations ({\sc{ciao}}) version 4.15 \citep{2006SPIE.6270E..1VF}, with calibration data base version 4.9.8. In particular, we created a new level-2 event file with the {\sc{ciao}} task {\texttt{chandra\_repro}} and filter it in energy and time with {\texttt{dmcopy}}. We obtained the sky position in Table \ref{tab:nstable} using the {\sc{ciao}} tool \texttt{wavdetect}. To reduce background noise and improve the determination of the source centroid, we applied \texttt{wavdetect} on an image filtered to include only the time interval from the beginning of the flare ($t_0$) until a time $t_0 + 920$\,s. The 90\% uncertainty radius of 2.0\,arcsec is the combination of the uncertainty in the source centroid position reported by \texttt{wavdetect}, and the absolute astrometry uncertainty in a typical ACIS observation for off-axis sources}.

The field was previously covered by four other \emph{Chandra} observations (ObsIDs 776, 7635, 8554, and 23023) with no source detections at the location of 2CXO J051117.2$-$672556. We estimated model-independent upper limits to the source flux and luminosity with {\sc{ciao}} tool \texttt{srcflux}. In the pre-flare part of ObsID 23022, we obtained a 90\% confidence limit of $L_{\rm{X}}<1.0\times10^{34}\,\mathrm{erg/s}$ in the 0.3--7\,keV band at the LMC distance of 50\,kpc. Stacking the data from all the ObsIDs with non-detections, including the pre-flare part of ObsID 23022, results in a total observed exposure of approximately $\sim$150\,ks, and yields a 90\% confidence upper limit on the X-ray luminosity is $L_{\rm{X}}<3\times10^{33}\,\mathrm{erg/s}$.

 \begin{table}
\centering
\caption{Properties of the \emph{Chandra} observation ObsID 23022 and source 2CXO J051117.2$-$672556 associated with XRT\,200515.}
\begin{tabular}{ll}
\hline
ObsID $23022$ & \\
\hline
Observation start time (UTC) & $2020$-$05$-$15$ 11:45:37\\
Exposure [ks] & 25.06 \\
\hline
XRT\,200515 (2CXO J051117.2$-$672556) & \\
\hline
Flare start time $t_0$ (UTC) & $2020$-$05$-$15$ 18:36:46 \\
R.A., Dec. (J2000) & 5:11:17.17 $-$67:25:55.9 \\
90\% position error radius [arcsec] & $2.0$\\
Off-axis angle [arcmin] & $11.75$\\
S/N & $11.64$\\
\hline
\end{tabular}
\label{tab:nstable}
\end{table}

 \begin{figure}
 \centering
\includegraphics[width=1\columnwidth]{figures/xrt_image.png}
 \caption{ACIS-S image for \emph{Chandra} observation ObsID 23022 showing the target, SNR 0509$-$67.5, on the bottom right and the transient event, XRT\,200515, on the top left. For this image, the event file was filtered to include only the time interval $t_0 + 920$\,s around the flare. Red counts correspond to photons in the 0.3--1.2\,keV band, yellow counts correspond to photons in the 1.2--2.4\,keV band, and blue photons correspond to photons in the 2.4--7\,keV band. The inset image is a 1.0\,arcmin $\times$ 1.0\,arcmin zoomed-in view. The dashed ellipse has semi-minor and semi-major axes of 15\,arcsec $\times$ 20\,arcsec, and is the source region used for spectral extraction.}
 \label{Fig:xrt_in_sky}
\end{figure}

\subsubsection{X-ray Temporal Analysis}

We used the {\sc{ciao}} tool {\texttt{dmextract}} to extract background-subtracted lightcurves in several energy bands, from the reprocessed event file of {\it{Chandra}} ObsID 23022. We defined an elliptical source extraction region, with semi-minor and semi-major axes of 15\,arcsec and 20\,arcsec (matching the point-source PSF at the source location); the local background region was chosen in the same ACIS chip, with an area approximately eight times larger.

Figure \ref{Fig:lightcurve_raw} shows the 0.3--7 keV background-subtracted lightcurve of XRT\,200515 with a time resolution of 20\,s. The lightcurve is consistent with no source detection at the location of the transient, before the start of the flare at around 23.5\,ks into the observation. The few pre-flare counts are consistent with background noise. The lightcurve exhibits a strong initial spike with a sharp rise of at least 4 orders of magnitude in $<$10\,s, containing 44 out of all $\sim$180 flare counts. This initial burst is followed by a sudden drop to a $\sim$800\,s long pulsating and decaying tail. We estimate a $T_{90} \sim $ 580--740\,s for the photons observed in the 0.3--7\,keV band$ is the time interval during which the cumulative number of counts increases from 5\% to 95\% of the total flare counts \citep{1993ApJ...413L.101K}.}, depending on the definition of total flare counts.

Figure \ref{Fig:tempfit} shows the lightcurve of XRT\,200515 at a resolution matching the ACIS frame time of 3.2\,s, the hardness ratio, and the energy evolution for the time interval $t_0 + 920$\,s. The lightcurve exhibits a spike in the count rate across only 3 bins (with a total of 4, 31 and 9 counts, respectively), hence the burst duration of $<$10\,s. The rise and fall times of the burst are both between 3.2\,s and 6.4\,s. The maximum count rate at the \emph{Chandra} frame time resolution is $\sim$9.7\,counts/s, acting as the lower bound for the peak count rate of the burst. Those counts are spatially spread over a PSF area of $\sim$3000 pixels; therefore, pile-up is not an issue. We evaluated the hardness ratio evolution during the flare with the Bayesian estimation method \texttt{BEHR} \citep{2006ApJ...652..610P}. Here, the hardness ratio is defined as:

\begin{equation}
HR = \frac{h - m - s}{h + m + s},
\end{equation}
where $s$ is the number of soft photons (0.3--1.2\,keV), $m$ is the number of medium photons (1.2--2\,keV), and
$h$ is the number of hard photons (2--7\,keV) in each bin. We also track the running average of the photon energies during the flare with a moving window of $\pm10$\,counts. The hardness ratio and energy evolution indicate spectral softening during the flare, with the hardness ratio starting at 1 during the hard burst peak and decreasing to a range of 0.4 to 0.6 in the tail, highlighting the notable spectral variability of XRT\,200515.
\begin{figure}
\centering
\includegraphics[width=1\columnwidth]{figures/lc_xrt_200515.pdf}

\caption{Background-subtracted lightcurve of XRT\,200515 in the 0.3--7\,keV energy range with a bin size of 20\,s. The zero start time is taken as the start of the \emph{Chandra} observation ObsID 23022, and the time interval shown is the full exposure time of 25.06\,ks. The inset shows the last $\sim$2\,ks of the observations, and captures the initial burst and tail of the transient event. The bin size is chosen to better visualize the oscillatory decay of the tail. The actual duration of the initial burst peak is $<$10\,s. The presence of a few negative counts arises from the process of background subtraction.}
\label{Fig:lightcurve_raw}
\end{figure}

 \begin{figure}
 \centering
 \includegraphics[width=1\columnwidth]{figures/energy_evolution.pdf}
 \caption{\textit{Upper panel:} Background-subtracted count rate lightcurve of XRT\,200515 in the 0.3--7\,keV energy range at the full \emph{Chandra} detector resolution of 3.2\,s and hardness ratio evolution obtained for a minimum of 20 counts per bin. The zero start time is taken as the flare start time $t_0$ of XRT\,200515, and the time interval shown is $t_0+920$\,s. The initial peak is $<$10\,s long and is very hard, while the $\sim$800\,s long oscillatory tail is significantly softer. \textit{Bottom panel:} The energy evolution during the flare obtained from the running average energy with a moving window of $\pm10$\,counts, showing significant spectral variability and softening during the flare. The scatter points represent the time and energy of individual photons from XRT\,200515 in the event file associated with \emph{Chandra} observation ObsID 23022.}
 \label{Fig:tempfit}
\end{figure}

\subsubsection{X-ray Spectral Analysis} \label{sec:specana}

We used the {\sc{ciao}} tool {\texttt{specextract}} to extract the spectrum and the associated response and ancillary response files from the reprocessed event file of {\it{Chandra}} ObsID 23022. We used the same source and background extraction regions defined for the lightcurve extraction. To improve the signal-to-noise ratio of the source, we extracted the spectrum only from the time interval $t_0 + 920$\,s. We binned the spectrum to a minimum of 1 count per bin with the {\texttt{grppha}} task within the {\sc{ftools}} package suite \citep{blackburn95} from NASA's High Energy Astrophysics Science Archive Research Center (HEASARC)}. For all spectral modelling and flux estimates, we used the {\sc{xspec}} software version 12.13.0 \citep{arnaud1996astronomical}. With only 179 net counts, we are unable to fit complex spectral models; thus, we limit our analysis to the simplest one-component models representative of opposite scenarios: a power law (\texttt{powerlaw}) and a blackbody model (\texttt{bbody}), both modified by photo-electric absorption (\texttt{tbabs}). In both cases, we adopted the Tuebingen-Boulder absorption model with Wilms abundances \citep{wilms00}. We minimized the Cash statistic \citep{1979ApJ...228..939C}, as we do not have enough counts for $\chi^2$ fitting.

The best-fitting power-law model (Table \ref{table:specfit} and Figure \ref{Fig:spectralfit10}) has a photon index of $\Gamma=[NUMERICAL_RESULT]\pm0.3$. The fit statistics yield a null hypothesis probability of
$3.5 \times 10^{-3}$, with a Cstat value of 132.7 for 137 degrees of freedom. For the blackbody model, the best-fitting temperature is $kT_{\rm{bb}} = [NUMERICAL_RESULT]\pm0.3$\,keV (Table \ref{table:specfit}). The fit statistics yield a null hypothesis probability of
$1.2 \times 10^{-2}$, with a Cstat value of 129.6 for 137 degrees of freedom. The reason this blackbody spectrum may appear hard in the {\it{Chandra}} band, resembling a $\Gamma\sim0.5$ power law, is that at a temperature of $kT_{\rm{bb}}\sim[NUMERICAL_RESULT]$\,keV, the ACIS detector samples only the peak and the Rayleigh-Jeans (rising) portion of the blackbody emission. We can use either model to determine an average conversion between the count rate and luminosity. This will then enable us to estimate the peak luminosity in the initial spike, for which we have previously estimated a peak count rate of $\gtrsim$10\,counts/s. The best-fitting power law model implies a peak flux of $F_{\rm{p}} \gtrsim 5.6 \times 10^{-10}$\,erg/s/cm$^2$, a total flare fluence of $E_{\rm{f}} \gtrsim 1.1 \times 10^{-8}$\,erg/cm$^2$, and a peak unabsorbed 0.3--10\,keV luminosity of $L_{\rm X}\gtrsim1.7$ $\times$ $10^{38}$\,erg/s at the LMC distance of 50\,kpc. For the best-fitting blackbody model, the peak flux and flare fluence would be $F_{\rm{p}} \gtrsim 4.0 \times 10^{-10}$\,erg/s/cm$^2$ and $E_{\rm{f}} \gtrsim 0.8 \times 10^{-8}$\,erg/cm$^2$ respectively. The peak unabsorbed 0.3--10\,keV luminosity would be $L_{\rm X}\gtrsim1.2$ $\times$ $10^{38}$\,erg/s and the peak bolometric luminosity would be $L_{\rm bol}\gtrsim1.5 \times 10^{38}$\,erg/s. These values should be considered conservative lower limits for two reasons: (i) the peak count rate provides only a lower bound estimate, as it is constrained by the \emph{Chandra} frame time resolution of the observations, potentially underestimating the true peak count rate; and (ii) the conversion factor applied is derived from the average spectrum over the entire flare, even though the spectrum of the initial spike is significantly harder compared to the tail, as shown in Figure \ref{Fig:tempfit}.

\subsubsection{High-energy Counterpart Search}

We searched for potential detections of XRT\,200515 by other high-energy facilities. However, no significant X-ray or $\gamma$-ray events in the field around the X-ray source coordinates and flare start time $t_0$ reported in Table \ref{tab:nstable} were detected by the Fermi Gamma-ray Space Telescope (\emph{Fermi}), the Burst Alert Telescope (\emph{BAT}) on the Neil Gehrels Swift Observatory (\emph{Swift}), the International Gamma-Ray Astrophysics Laboratory (\emph{INTEGRAL}), or the Monitor of All-sky X-ray Image \emph{MAXI}. \emph{LIGO} was not operational during the time of the FXT, hence no gravitational wave signal could have been detected if the origin of XRT\,200515 was a compact object merger.

\subsubsection{Optical Counterpart Search}

We used the X-ray source coordinates reported in Table \ref{tab:nstable} to search for optical counterparts to XRT\,200515. The field of XRT\,200515 was covered by the Survey of Magellanic Stellar History (\emph{SMASH}) \citep{2017AJ....154..199N}, a deep optical survey in the \emph{ugriz} bands with the Dark Energy Camera (DECam) mounted on the Víctor M. Blanco Telescope at the Cerro Tololo Inter-American Observatory (CTIO) in Chile. We used the Astro Data Lab Jupyter Notebook server \citep{2020A&C....3300411N, 2021CSE....23b..15J} to access and visualize the \emph{SMASH} catalog}. Figure \ref{Fig:smash} shows a color image of the field created from the deepest available stacked images in the \emph{u}, \emph{g} and \emph{i} bands; the 5$\sigma$ detection limits in these bands are 23.9\,mag, 24.8\,mag and 24.2\,mag, respectively. The images were taken on December 7, 2015 with exposure times of 1,179\,s, 981\,s and 1,179\,s respectively. The astrometry of the \emph{SMASH} images is calibrated on the Gaia DR3 reference frame, thus their positional uncertainty is negligible compared to the X-ray source position uncertainty. Within the \emph{Chandra} position error circle in Figure \ref{Fig:smash}, there is no obvious optical counterpart that stands out in brightness or color from the surrounding stellar population. We performed relative photometry on the sources inside the error circle, comparing them to several nearby sources with known positions and brightnesses listed in the Gaia DR3 catalog. We used the \emph{SMASH} \emph{g} band as the closest approximation to Gaia's \emph{G} band. We estimate the brightest optical source within the error circle to have a Vega magnitude of $g=22.7\pm0.1\,\mathrm{mag}$, corresponding to an absolute magnitude of $M_{g}\approx4.2$, assuming it is in the LMC. Additionally, three other point-like sources are detected with $g$ band magnitudes in the range of 23--24\,mag. All four sources appear point-like, consistent with the seeing conditions of the \emph{SMASH} survey, with no evidence of any spatially extended background galaxies. The three brightest stars visible in Figure \ref{Fig:smash} within $\sim$12\,arcsec of the \emph{Chandra} source are solar-mass stars on the red giant branch, indicative of an old stellar population. The lack of bright optical counterparts and the short burst duration of $<$10\,s rules out a stellar flare from a foreground Galactic low-mass star \citep{2004A&ARv..12...71G, 2007A&A...471..271R,2012A&A...543A..90R,2015A&A...581A..28P,2021ApJ...912...81K}. A flare from a Be/X-ray binary or any other HMXB in the LMC is also excluded by the lack of a bright optical counterpart \citep{2019ApJ...881L..17D,2022A&A...661A..22D}. The temporal and spectral properties of XRT\,200515, combined with the absence of an optical counterpart, suggest three possibilities: (i) a relativistic jet phenomenon, such as a $\gamma$-ray burst (GRB); (ii) a rapid, high-energy process linked to extreme magnetic fields, such as a giant magnetar flare (GMF); or (iii) a thermonuclear Type I X-ray burst caused by surface nuclear burning on a neutron star.

 \begin{figure}
 \centering
 \includegraphics[width=1\columnwidth]{figures/color_image.png}
 \caption{\emph{SMASH} survey color image of the field of XRT\,200515 created from the deepest available stacked images in the \emph{u}, \emph{g} an \emph{i} bands. Red corresponds to the \emph{i} band, green to the \emph{g} band, and blue to the \emph{u} band. The dashed circle has a radius of 2\,arcsec, and is the 90\% position uncertainty of the \emph{Chandra} source.}
 \label{Fig:smash}
\end{figure}

\subsubsection{Gamma Ray Burst from a Compact Object Merger?}

Evidence in favor or against the association of at least some \emph{Chandra} FXTs with low-luminosity long-GRBs or off-axis short-GRBs (see \citealt{berger14} for a review), at moderate or high redshifts, is extensively discussed in \cite{quirola2022extragalactic}, \cite{2023arXiv230413795Q}, and \cite{wichern24}. A detailed re-investigation of this issue is beyond the scope of this work. Here, we simply point out that XRT\,200515, like the other \emph{Chandra} FXTs in the literature, does not have any $\gamma$-ray detection. On the other hand, XRT\,200515 has a significantly harder spectrum ([NUMERICAL_RESULT]) in the \emph{Chandra} band than the rest of the FXT sample, all of which have photon indices of $\Gamma>1$ \citep{2013ApJ...779...14J,2015MNRAS.450.3765G,2017MNRAS.467.4841B, 2019Natur.568..198X,2022ApJ...927..211L,quirola2022extragalactic,2023arXiv230413795Q,2023ApJ...948...91E}. A photon index of [NUMERICAL_RESULT] below 10\,keV is indeed expected and observed from both core-collapse GRBs and compact-merger GRBs \citep{ghirlanda09,bromberg13,2018A&A...616A.138O, 2019A&A...625A..60R,2021A&A...652A.123T}. This might support the association of the initial spike of XRT\,200515 with a GRB. However, the presence and properties of the $\sim$800\,s tail (candidate GRB afterglow) is puzzling. The $T_{90} \sim $ 580--740\,s value for XRT\,200515 is significantly shorter than in most other \emph{Chandra} FXTs \citep{quirola2022extragalactic,2022ApJ...927..211L,2023arXiv230413795Q}, which have $T_{90}$ values on the order of several ks and are already pushing the limit for a GRB afterglow detection \citep{wichern24}. Moreover, XRT\,200515's initial burst duration ($<$10\,s), its short rise and fall times (3.2-6.4\,s), and the lack of a peak plateau are inconsistent with the lightcurves of \emph{Chandra} FXTs interpreted as magnetar-powered GRBs as the aftermath of a binary neutron star merger, such as CDF-S XT1 \citep{2017MNRAS.467.4841B}, CDF-S XT2 \citep{2019Natur.568..198X} and the sample in \cite{2022ApJ...927..211L}. Finally, the lack of any optical evidence for a host galaxy is another element disfavoring the high-redshift GRB interpretation.

\subsubsection{Giant Magnetar Flare from a Soft Gamma Repeater?}

Based on its temporal and spectral variability, it is tempting to interpret XRT\,200515 as a rare GMF from a SGR \citep{2008A&ARv..15..225M, 2015RPPh...78k6901T} in the LMC or behind it, which can easily explain the burst's strong increase of at least 4 orders of magnitude in $<$10\,s \citep{2018MNRAS.474..961C}. Similar to XRT\,200515, GMFs are characterized by a short and hard initial spike and a longer and softer, pulsating tail. GMFs are extremely rare, with only a select few ever discovered. Well-studied examples are SGR 0526$-$66 in the LMC \citep{1979Natur.282..587M}, and the Galactic sources SGR 1900$+$14 \citep{1999Natur.397...41H} and SGR 1806$+$20 \citep{2005Natur.434.1098H, 2005Natur.434.1107P, 2005ApJ...628L..53I}. More recently, GMFs have been identified in M\,31 \citep{mazets08}, NGC\,253 \citep{2021NatAs...5..385F, 2021Natur.589..211S,2021Natur.589..207R,trigg24} and M\,82 \citep{2024Natur.629...58M}. All of these have been observed by high time resolution instruments in the hard X-rays and soft $\gamma$-rays with luminosities above $10^{46}$\,erg/s for a fraction of a second in the initial spike. The tails of GMFs are often modulated by magnetar spin periods of 2--12\,s, leading to quasi-periodic oscillations (QPOs). For XRT\,200515, there is no hard X-ray or $\gamma$-ray detection, despite the LMC direction being in good visibility for most of the previously mentioned high-energy facilities. We were unable to identify any significant periodicities in the tail of XRT\,200515 through periodogram analysis, which is unsurprising given the low time resolution of \emph{Chandra} observations. No X-ray activity has been observed by \emph{Chandra} or other X-ray telescopes in the years before or after XRT\,200515, which may be because SGRs are very faint when they are not bursting. The strongest argument against a magnetar in the LMC as the origin of XRT\,200515 is that magnetars are short-lived objects ($\lesssim$10$^5$\,yr) associated to young stellar populations \citep{olausen14,nakano15,mondal21}. Even allowing for the persistence of magnetar-like activity in ordinary radio pulsars as old as $\sim$10$^7$\,yr \citep{rea10}, this scenario is still inconsistent with the old stellar population (several Gyr) in the LMC field shown in Figure \ref{Fig:smash}. The nearest star-forming regions in the LMC are $\sim$10\,arcmin ($\sim$150\,pc) away. If (in a very contrived scenario), we assume that XRT\,200515 is powered by a young neutron star ejected from one of those regions, we estimate a characteristic time of 1\,Myr to travel that distance at a speed of 150\,km/s. Therefore, if XRT\,200515 is a GMF, it must be located behind the LMC, in a low-redshift galaxy \citep{2005Natur.434.1098H,2005Natur.438..991T}. Since GMFs have been observed only a few times and never at soft X-ray energies, their properties in the soft X-ray band detectable by \emph{Chandra} remain largely unexplored. XRT\,200515 could indeed be the first GMF detected at soft X-ray energies. Distinguishing distant short GRBs from GMFs has historically been difficult and there are multiple studies suggesting that a subset of short GRBs are actually extragalactic GMFs \citep{2005Natur.434.1098H, 2005Natur.434.1107P, 2005Natur.438..991T, 2006ApJ...652..507O, 2008ApJ...680..545M, 2011AdSpR..47.1337H, 2020ApJ...899..106Y, 2021Natur.589..211S, 2023IAUS..363..284N}. Just as for the distant GRB interpretation, the non-detection of any optical counterpart remains puzzling for a distant GMF scenario, unless we are dealing with a very distant and exceptionally luminous GMF.

\subsubsection{Thermonuclear X-ray Burst from a quiet LMXB in the LMC?}

If XRT\,200515 is in the LMC, a peak luminosity near the Eddington luminosity $L_{\rm{Edd}}\sim10^{38}$\,erg/s and sharp rise time of the flare suggests a Type I X-ray burst interpretation, which is a thermonuclear explosion on the surface of a weakly magnetized, accreting neutron star \citep{lewin93,strohmayer06,galloway08,galloway20,galloway21,alizai23}. The old stellar population in the field of XRT\,200515 is consistent with the presence of neutron star LMXBs. Following the definition of burst timescale $\tau = E_{\rm f}/F_ {\rm p}$ in \cite{galloway08}, we estimate $\tau \sim 20$\,s for XRT\,200515, which is consistent with Type I X-ray bursts \citep{galloway21,alizai23}. The fitted temperature $kT_{\rm{bb}} \sim [NUMERICAL_RESULT]$\,keV when the average spectrum is fitted with a simple blackbody, and the softening of the spectrum (temperature decrease) in the tail is also typical of Type I X-ray bursts \citep{galloway08,galloway20, guver12}. On the other hand, several observed properties of XRT\,200515 are unusual for Type I X-ray bursts. In particular, most Type I X-ray bursts occur when the persistent luminosity (proportional to the accretion rate) of a LMXB is $L_{\rm{X}} >10^{-4} L_{\rm{Edd}}$ (and, in most cases, $L_{\rm{X}}>10^{-3} L_{\rm{Edd}}$) \citep{galloway08}. Instead, in the initial part of ObsID 23022, the upper limit on the X-ray luminosity at the position of XRT\,200515 is
$L_{\rm{X}}<10^{-4} L_{\rm{Edd}}$, so that the X-ray flux increased by at least 4 orders of magnitudes. On another note, the sharp decline after the initial burst of XRT\,200515 would be unusual for Type I X-ray bursts, which typically exhibit a gradual and exponential decay. However, note that most Type I X-ray bursters were observed by the Rossi X-Ray Timing Explorer (\emph{RXTE}) \citep{1996SPIE.2808...59J}, which has a high time resolution. The low time resolution of \emph{Chandra} may have obscured such a decay for XRT\,200515. Moreover, most Type I bursts tend to repeat every few hours \citep{galloway08}; instead, XRT\,200515 is the only event detected at that location over a total observed time of $\sim$150\,ks. No LMXB has ever been noted at that position before or after the event. The time interval between bursts is related to an index $\alpha$ defined as the ratio between the integrated persistent fluence between subsequent bursts and the burst fluence; from a comparison of the energy released by accretion (contributing to the persistent fluence) and by thermonuclear burning (burst fluence), we expect $\alpha \gtrsim 40$, in agreement with the observations of Type I bursts \citep{galloway08}. If we apply the same criterion ($\alpha \gtrsim 40$) to the persistent and flare fluences of XRT\,200515, we would have to wait $>$10$^{7}$\,s (4 months) to observe another similar event, assuming the persistent flux level upper limit in ObsID 23022 before the transient event. This waiting time extends to at least one year if we assume the persistent flux upper limit derived from the stacked $\sim$150\,ks \emph{Chandra} observations. Only a few one-off bursts from Galactic neutron stars at a very low persistent luminosity ($L_{\rm X} \sim 10^{32}$--$10^{33}$\,erg/s) were found by \cite{{cornelisse02a,cornelisse02b}} with estimated recurrence times of tens of years. The vast majority of Type I X-ray bursts are Galactic, due to their lower flux at large distances. Only a handful of extragalactic Type I X-ray bursts are documented, for example in M\,31 \citep{2020A&A...640A.124P} and the Magellanic Bridge \citep{2023A&A...669A..66H}. If XRT\,200515 is a Type I X-ray burst, it is the first extragalactic Type I X-ray burster in the LMC and represents the tip of the iceberg for a vast population of faint LMXBs in nearby galaxies, too dim to be detected by {\it{Chandra}} or {\it{XMM-Newton}}, but which may occasionally reveal themselves via thermonuclear bursts with a long duty cycle.

\subsubsection{Concluding Remarks and Outlook for XRT\,200515}

XRT\,200515 is a unique and intriguing extragalactic $\emph{Chandra}$ FXT. The combination of its temporal and spectral properties is unlike any of the other $\emph{Chandra}$ FXT samples. Based on our analysis, the two most likely scenarios for XRT\,200515 are: (i) a distant GMF from a SGR behind the LMC; the first observed in the low X-ray energy band, missed by any other high-energy facilities, or (ii) an unusual Type I X-ray burst from a previously unknown faint LMXB; the first extragalactic X-ray burster in the LMC. Nevertheless, both of these interpretations come with their own unique challenges. XRT\,200515 could, in fact, represent an entirely new type of astronomical phenomenon. After all, the primary objective of our work was to use machine learning to find rare, needle-in-the-haystack anomalies hidden within vast astronomical datasets. We invite further detailed studies of XRT\,200515 to evaluate our interpretations and explore alternative scenarios, such as potential associations with a fast radio burst (FRB) or a SBO. We highly recommend follow-up multi-band observations at the source coordinates of XRT\,200515 to better constrain its nature. Lastly, we note that XRT\,200515 and the second transient discovered by \cite{2015MNRAS.450.3765G}, XRT\,120830, have remarkably similar temporal evolutions in their lightcurves (J. Irwin, personal communication, November 2024), however with very different spectral properties ($\Gamma\sim2.5$ for XRT\,120830 versus $\Gamma\sim[NUMERICAL_RESULT]$ for XRT\,200515). We leave a detailed comparative analysis of these transients for future work.

Figure \ref{Fig:23022_rep} shows the $E-t$\,Map and $E-t-dt$\,Cube event file representations for XRT\,200515. These exhibit high counts at high energies in a narrow time window, which is in line with the hard spectrum and transient nature of XRT\,200515.

\begin{figure}
 \centering

 \includegraphics[width=0.495\columnwidth]{figures/2dmap_23022_122.pdf}
 \includegraphics[width=0.495\columnwidth]{figures/3dmap_23022_122.pdf}
 \label{Fig:2d}
 \caption{$E-t$\,Map event file representation (left) and $E-t-dt$\,Cube event file representation (right) of XRT\,200515 in \emph{Chandra} observation ObsID 23022. The catalog identifier for XRT\,200515 is \texttt{CATALOG\_ID: 23022\_122}.}
 \label{Fig:23022_rep}
\end{figure}

\subsection{Technical Caveats}

The main technical caveat of our approach is related to the representation of event files.
While our new event file representations enable a simple, yet powerful representation learning approach to find new and rare X-ray transients, any simplification of raw event files, like the fixed number of time bins we use across all event files, is associated with a loss of information. This could lead to us missing a small amount of transients. To minimize this, we have implemented a rigorous approach to justify the resolution of the event file representations in \S~\ref{sec:datarepp}. Moreover, flares, in particular known extragalactic FXTs, cluster notably well in our representation spaces. This is because their distinctive features are less dependent on the temporal binning resolution in the $E-t$\,Maps and $E-t-dt$\,Cubes. To improve the effectiveness of dip searches with our proposed method, we suggest using higher resolution event file representations. Nevertheless, our comprehensive transient candidate catalog includes numerous newly identified transients that were previously overlooked by other X-ray transient searches in the \emph{Chandra} archive. Among these is the remarkable needle-in-the-haystack event XRT\,200515 discovered in this work, underscoring the effectiveness of our method. A follow-up representation learning algorithm will learn informative features from raw and unbinned event files while accounting for the Poisson nature of X-ray observations (Song et al., 2025, in preparation).

\section{Conclusion} \label{sec:discussion}

We have introduced a novel representation learning method, the first of its kind applied to X-ray event files, enabling downstream tasks such as unsupervised classification and anomaly detection in high-energy astrophysics. We have used the learned representation to investigate time-domain properties of sources in the \emph{Chandra} archive, with a particular emphasis on the discovery of X-ray transients. As a result, we have compiled the identified X-ray flares and dips in a comprehensive catalog of transient candidates. Notably, our method led to the discovery of XRT\,200515; a previously unidentified extragalactic FXT with unique temporal and spectral properties, representing a genuine needle-in-the-haystack discovery. Our key results are as follows:

\begin{enumerate}
 \item We introduce novel event file representations, the \textit{E-t}\,Maps and \textit{E-t-dt}\,Cubes, which capture both temporal and spectral information.

 \item We apply two feature extraction methods to the event file representations, PCA and sparse autoencoder neural networks, to extract or learn informative features that can be utilized for downstream tasks, such as unsupervised classification or anomaly detection.

 \item We project the learned features to two-dimensional embedding spaces, enabling interpretable queries of analogs to objects of interest based on their temporal and spectral properties.

 \item We cluster the embedding spaces with DBSCAN, successfully isolating previously identified X-ray transients. We identify new transient candidates within specific transient-dominant clusters or through nearest-neighbor searches using kNN.

 \item We compile a catalog of the X-ray transient candidates, including 3,447 flares and 112 dips, and make it openly accessible to the community and the broader scientific audience.

 \item We report the discovery of XRT\,200515, a rare extragalactic FXT characterized by unique temporal and spectral features. We explore its potential origins and suggest that it may be associated with one of the following scenarios, presented in no particular order:
 \begin{itemize}
 \item A rare GMF from an SGR behind the LMC, marking the first GMF detected in the low X-ray energy range covered by telescopes like \emph{Chandra}, \emph{XMM-Newton}, \emph{Swift-XRT}, \emph{eROSITA}, or \emph{Einstein Probe}.
 \item A rare extragalactic Type I X-ray burst from a faint LMXB in the LMC, representing the first such detection in the LMC.
 \item A new type of astronomical phenomenon and a genuine anomaly, previously hidden in the vast \emph{Chandra} archive.
 \end{itemize}
 XRT\,200515 was only detected by \emph{Chandra}, with no identified optical counterparts. We strongly encourage a multi-wavelength search for additional signals from the source associated with XRT\,200515 to better understand its origin and nature.
\end{enumerate}

Our work advances time-domain high-energy astrophysics by making the \emph{Chandra} transient candidates catalog publicly available and open-sourcing the representation learning based transient search pipeline{https://github.com/StevenDillmann/ml-xraytransients-mnras}}. The catalog enables queries to identify and characterize new \emph{Chandra} transients. Future work involves experimenting with different event file representations, applying the detection pipeline to additional high-energy archives, and adapting it to a variety of other scientific datasets, paving the way for further machine learning driven discoveries of rare transients and other scientific anomalies.

\section{Event File Lengths and Durations} \label{appendix:distributions}

Figure \ref{Fig:distr} shows the distribution of the length $N$ and duration $T$ of event files in the dataset used in this work.

\begin{figure}
\centering
\includegraphics[width=0.5\textwidth]{figures/n_dist.pdf}
\includegraphics[width=0.5\textwidth]{figures/t_dist.pdf}
 \caption{Distribution of \emph{Chandra} event file lengths $N$ (top) and durations $T$ (bottom) in the dataset used in this work.}
 \label{Fig:distr}
\end{figure}

\section{Autoencoder Training Process} \label{appendix:training}

Figure \ref{TRAINING} shows the training process of the autoencoders used in this work.

\section{Hyperparameter Optimization} \label{appendix:hyperparameters}

Below, we summarize the optimization strategy for the t-SNE and DBSCAN hyperparameters. For even more details on this approach, please refer to \cite{sdthesis}.

\subsection{t-SNE Hyperparameters}

The choice of the \texttt{perplexity} and \texttt{learning\_rate} can have a large impact on the resulting t-SNE embedding space. Ideally, we want the two-dimensional embedding space to effectively capture both energy information (hardness ratio $HR$) and variability information (variability probability $p_{\rm var}$). Therefore, event files with similar values for $HR$ and $p_{\rm var}$ should live close to each other in the embedding space. We can use this information to define a performance metric for different t-SNE hyperparameter inputs. First, we compute the pairwise distance matrix $\mathbf{D_{Z}}$ of size $(m,m)$, where the distance $D_{Z_{ij}}$ between points $i$ and $j$ is computed using a Euclidean distance metric. Next, we define the property vector $\mathbf{Y}$, which includes $7$ CSC properties (hardness ratios $HR_{\rm hm}$, $HR_{\rm hs}$, $HR_{\rm ms}$ and variability probabilities $p_{\rm var}^{\rm b}$, $p_{\rm var}^{\rm h}$, $p_{\rm var}^{\rm m}$, $p_{\rm var}^{\rm s}$) for each event file. As a measure of similarity between the properties of different points, we can again compute a pairwise similarity matrix $\mathbf{D_{Y}}$ of size $(m,m)$. To compute the similarity distance $D_{Y_{ij}}$ between sample $i$ and $j$, we use the Mahalanobis distance metric \citep{mahalanobis1936generalised}. Unlike the Euclidean distance metric, the Mahalanobis distance metric accounts for the correlation between different labels by taking into account the covariance structure of the data. Note that our hardness ratios are correlated with each other, and that the same holds for the variability probabilities. Accounting for these correlations provides a more accurate measure of the similarity distance between different samples. Having computed $\mathbf{D_{Z}}$ and $\mathbf{D_{Y}}$, we can define a performance metric that allows us to compare the performance of different t-SNE hyperparameters. The smaller the distance $D_{Z_{ij}}$ between two points $i$ and $j$ in the t-SNE embedding, the smaller should be difference in their associated labels as measured by the distance $D_{Y_{ij}}$. We can thus define a performance metric based on the statistical correlation of $\mathbf{D_{Z}}$ and $\mathbf{D_{Y}}$ using the Spearman’s rank correlation coefficient $\rho_{ZY}$ \citep{spearman1904proof}. The higher $\rho_{ZY}$, the higher is the positive correlation between $\mathbf{D_{Z}}$ and $\mathbf{D_{Y}}$ and the better the performance of the t-SNE embedding.
The hyperparameter space is given by the ranges \texttt{learning\_rate} \in (20,200) with a step size of 20 and \texttt{perplexity} \in (10,100) with a step size of 10. This optimization process is performed using a reduced dataset of 15,353 samples for 2,000 iterations per hyperparameter combination due to computational constraints. While subsampling, the overall structure of the data was preserved by selecting the same distributions between any combinations of hard, medium, soft, variable and non-variable samples. This ensures that the sample set is representative of the original data. We choose the hyperparameter combination that produces the highest value of $\rho_{ZY}$.
\subsection{DBSCAN Hyperparameters}

Different hyperparameter combinations of \texttt{eps} and \texttt{minPts} can have a large impact on the resulting DBSCAN clusters. We use a combination of the Davies-Bouldin index $DB$ \citep{davies1979cluster} and Calinski-Harabasz index $CH$ \citep{calinski1974dendrite} as a performance metric to find the optimal DBSCAN hyperparameter inputs. The $DB$ index is a measure of the average similarity between each cluster and its most similar cluster, relative to the average distance between points within each cluster. The $DB$ index is given by the following formula:
\begin{equation}
DB = \frac{1}{n_c} \sum_{i=1}^{n_c} \max_{j \neq i} \left(\frac{W_i + W_j}{d(c_i, c_j)}\right),
\label{DB}
\end{equation}
where $n_c$ is the number of clusters, $W_i$ and $W_j$ are the within-cluster sum of squares for cluster $i$ and $j$, and $d(c_i, c_j)$ is the distance between the centroids of clusters $i$ and $j$. On the other hand, the $CH$ index is based on the concept that good clusters should have high intra-cluster similarity (cohesion) measured by the between-cluster dispersion $B$ and low inter-cluster similarity (separation) measured by the within-cluster dispersion $W$. $B$ is the sum of the pairwise distances between cluster centroids, and $W$ is the sum of the pairwise distances between points within each cluster. The $CH$ index is given by the following formula:
\begin{equation}
CH = \frac{B}{W} \times \frac{m - n_c}{n_c - 1},
\label{CH}
\end{equation}
where the scaling factor $\frac{m-n_c}{n_c-1}$ accounts for the total number of data points $m$ and the number of clusters $n_c$. A lower $DB$ index and higher $CH$ index indicate that the clustering algorithm is more effective in grouping similar data points together and separating different data points into distinct clusters. We thus define the performance metric $\rho_{DC}$ as the ratio of the normalized indices $DB_n = \frac{DB}{\text{max}(DB)}$ and $CH_n = \frac{CH}{\text{max}(CH)}$ in the hyperparameter space given by $\texttt{eps} \in (1.0,3.0)$ with a step size of $0.1$ and $\texttt{minPts} \in (10,30)$ with a step size of $1$:
 \begin{equation}
\rho_{DBSCAN} = \frac{CH_n}{DB_n}.
\end{equation}

We choose the hyperparameter combination that produces the highest value of $\rho_{DBSCAN}$.

\section{Embeddings} \label{appendix:embeddings}

Figures \ref{Fig:tsneHAR2}, \ref{Fig:tsneVAR2} and \ref{fig:dbscanClusters2}
show the \textit{2D-PCA} and \textit{2D-AE} embeddedings.

\section{Transient-dominant Clusters} \label{appendix:transdomclus}
Table \ref{tab:candidate_selection_f} lists the transient-dominant clusters in the different embedding spaces used for the selection of transient candidates.

\begin{table}
\centering
\caption{Transient-dominant clusters used to find new transient candidates.}
\begin{tabular}{lll}
\hline
Case & Flares & Dips\\
\hline
\it{2D-PCA} & 3, 6, 10, 15, 18, 19, 22, 23, 29, 31, 33, 36, 37 & 12, 35\\
\it{3D-PCA} & 2-4, 6, 10, 11, 13-15, 17-19, 21, 22, 24 & -\\
\it{2D-AE} & 5, 6, 8, 10, 11, 14, 17, 21, 25, 26, 30& 9\\
\it{3D-AE} & 8& 4\\
\hline
\end{tabular}
\label{tab:candidate_selection_f}
\end{table}

\section{Catalog Columns} \label{appendix:columns}

Table \ref{tab:columns} shows the column descriptions of the X-ray transient candidates catalog.

\begin{table*}
\scriptsize
\centering
\caption{Column descriptions of the catalog of X-ray transient candidates found in this work.}
\label{tab:columns}
\begin{tabular}{ll}
\hline
Column Name & Column Description\\
\hline
\texttt{CATALOG\_ID} & Catalog identifier for the region event file obtained from the observation ID and region ID in the CSC\\
\texttt{CSC\_name} & Name of the source in the CSC\\
\texttt{TRANSIENT\_TYPE} & Transient type (F for flare candidate, D for dip candidate)\\
\texttt{CSC\_ra}& Right ascension of the source in the CSC [ $^\circ$]\\
\texttt{CSC\_dec} & Declination of the source in the CSC [ $^\circ$]\\
\texttt{CSC\_obs\_id} & Observation ID of the observation in the CSC\\
\texttt{CSC\_region\_id} & Region ID of the source in the CSC\\
\texttt{CSC\_significance} & Highest flux significance in any energy band in the CSC\\
\texttt{CSC\_flux\_aper\_b} & Background subtracted and aperture corrected broad band flux in the CSC\\
\texttt{CSC\_hr\_hm} & Hard-to-medium energy band hardness ratio as defined in the CSC\\
\texttt{CSC\_hr\_ms} & Medium-to-soft energy band hardness ratio as defined in the CSC\\
\texttt{CSC\_hr\_hs} & Hard-to-soft energy band hardness ratio as defined in the CSC\\
\texttt{CSC\_var\_prob\_b} & Variability probability in the broad energy band as defined in the CSC\\
\texttt{CSC\_var\_prob\_h} & Variability probability in the hard energy band as defined in the CSC\\
\texttt{CSC\_var\_prob\_m} & Variability probability in the medium energy band as defined in the CSC\\
\texttt{CSC\_var\_prob\_s} & Variability probability in the soft energy band as defined in the CSC\\
\texttt{CSC\_var\_index\_b} & Variability index in the broad energy band as defined in the CSC \\
\texttt{CSC\_gti\_obs} & Start date and time of the source observation in the CSC \\
\texttt{CSC\_gti\_end} & End date and time of the source observation in the CSC\\
\texttt{CSC\_theta} & Off-axis angle of the source in the CSC [arcmin] \\
\texttt{CSC\_cnts\_aper\_b} & Total counts measured in the modified source region for the broad energy band in the CSC\\
\texttt{CSC\_cnts\_aperbkg\_b} & Total counts measured in the modified background region for the broad energy band in the CSC\\
\texttt{CSC\_type} & Source type in the CSC (P for point sources, X for extended compact sources)\\
\texttt{CSC2.1\_flag} & Flag for processing status in the CSC (True for CSC 2.1 processing completed, False for only CSC 2.0 processing completed)\\
\texttt{PS\_id} & PanSTARRS-1 DR2 unique object identifier \\
\texttt{PS\_internal\_id} & PanSTARRS-1 DR2 internal object identifier\\
\texttt{sep\_CSC\_PS} & Angular distance between CSC and PanSTARRS-1 DR2 sources [arcsec] \\
\texttt{PS\_g} & PanSTARRS-1 $g$ magnitude [mag] \\
\texttt{PS\_r} & PanSTARRS-1 $r$ magnitude [mag] \\
\texttt{PS\_i} & PanSTARRS-1 $i$ magnitude [mag] \\
\texttt{PS\_z} & PanSTARRS-1 $z$ magnitude [mag] \\
\texttt{GAIA\_id} & Gaia unique object identifier\\
\texttt{sep\_CSC\_GAIA} & Angular distance between CSC and Gaia DR3 sources [arcsec] \\
\texttt{GAIA\_g} & Gaia $G$ magnitude [mag] \\
\texttt{GAIA\_rv} & Gaia radial velocity [km/s]\\
\texttt{GAIA\_parallax} & Gaia parallax [mas]\\
\texttt{GAIA\_bp\_rp} & Difference in magnitude between the blue photometer (BP) and the red photometer (RP) measurements in Gaia [mag] \\
\texttt{LS\_id} & Legacy Survey DR10 identifier\\
\texttt{LS\_object\_id} & Legacy Survey DR10 object identifier\\
\texttt{sep\_CSC\_LS} & Angular distance between CSC2.1 and Legacy Survey DR10 sources [arcsec]\\
\texttt{LS\_g} & Legacy Survey DR10 $g$ magnitude [mag]\\
\texttt{LS\_r} & Legacy Survey DR10 $r$ magnitude [mag]\\
\texttt{LS\_z} & Legacy Survey DR10 $z$ magnitude [mag]\\
\texttt{2MASS\_id} & 2MASS name\\
\texttt{sep\_CSC\_2MASS} & Angular distance between CSC and 2MASS sources [arcsec]\\
\texttt{2MASS\_j} & 2MASS $J$ magnitude [mag]\\
\texttt{2MASS\_h} & 2MASS $H$ mag [mag]\\
\texttt{2MASS\_k} & 2MASS $K$ mag [mag]\\
\texttt{SPEC\_group\_id} & Group ID for SDSS DR17 spectra with multiple CSC matches\\
\texttt{sep\_CSC\_SPEC} & Angular distance between CSC and SDSS DR17 spectroscopic sources [arcsec] \\
\texttt{SPEC\_class} & SDSS DR17 spectroscopic class (GALAXY, QSO, or STAR)\\
\texttt{SPEC\_subclass} & SDSS DR17 spectroscopic subclass\\
\texttt{SPEC\_z} & SDSS DR17 spectroscopy redshift\\
\texttt{SPEC\_zerr} & SDSS DR17 spectroscopy redshift error\\
\texttt{SPEC\_wavemin} & SDSS DR17 spectroscopy minimum observed (vacuum) wavelength [Angstroms]\\
\texttt{SPEC\_wavemax} & SDSS DR17 spectroscopy maximum observed (vacuum) wavelength [Angstroms]\\
\texttt{SPEC\_sn} & SDSS DR17 spectroscopy median signal-to-noise over all good pixels\\
\texttt{SPEC\_group\_size} & Number SDSS DR17spectra matched to CSC2.1 source in the group \\
\texttt{SDSS\_id} & SDSS DR15 object identifier\\
\texttt{sep\_CSC\_SDSS} & Angular distance between CSC and SDSS DR15 sources [arcsec] \\
\texttt{SDSS\_type} & SDSS DR15 type classification (S for stars, G for galaxies)\\
\texttt{SIMBAD\_ids} & SIMBAD object identifier or catalog names\\
\texttt{sep\_CSC\_SIMBAD} & Angular distance between CSC and SIMBAD sources [arcsec] \\
\texttt{SIMBAD\_otype} & SIMBAD object type classification\\
\texttt{SIMBAD\_sptype} & SIMBAD spectral type of the object \\
\texttt{SIMBAD\_parallax} & SIMBAD parallax [mas]\\
\texttt{SIMBAD\_pmra} & SIMBAD proper motion in right ascension [mas/yr]\\
\texttt{SIMBAD\_pmdec} & SIMBAD proper motion in declination [mas/yr]\\
\texttt{SIMBAD\_rv} & SIMBAD radial velocity [km/s]\\
\texttt{SIMBAD\_z} & SIMBAD radial redshift\\
\hline
\end{tabular}
\label{tab:catalog}
\end{table*}

\bsp
\label{lastpage}
\end{document}