Search for pair production of scalar leptoquarks decaying into first- or second-generation leptons and top quarks in proton–proton collisions at $\sqrt{s}$ = 13 TeV with the ATLAS detector

Abstract

A search for pair production of scalar leptoquarks, each decaying into either an electron or a muon and a top quark, is presented. This is the first leptoquark search using ATLAS data to investigate top-philic cross-generational couplings that could provide explanations for recently observed anomalies in B meson decays. This analysis targets high leptoquark masses which cause the decay products of each resultant top quark to be contained within a single high-$p_{\mathrm {T}}$ large-radius jet. The full Run 2 dataset is exploited, consisting of $139~\hbox {fb}^{-1}$ of data collected from proton–proton collisions at $\sqrt{s}=13~\mathrm {TeV}$ from 2015 to 2018 with the ATLAS detector at the CERN Large Hadron Collider. In the absence of any significant deviation from the background expectation, lower limits on the leptoquark masses are set at $1480~\mathrm {GeV}$ and $1470~\mathrm {GeV}$ for the electron and muon channel, respectively.

1 Introduction

The quark and lepton sectors of the standard model (SM) are interestingly similar, motivating one to hypothesize a fundamental symmetry between the two sectors. Such a symmetry can be found in many grand unified theories, such as grand unified SU(5) [1], the Pati–Salam model based on SU(4) [2], or R-parity-violating (RPV) supersymmetry (SUSY) models [3]. These models predict a new class of bosons carrying both lepton and baryon number, called leptoquarks (LQs). LQs are hypothetical colour-triplet bosons which couple directly to quarks and leptons. They can be of either scalar or vector nature, and carry fractional electric charge. The production cross section of vector LQs could be enhanced relative to that of scalar LQs due to the existence of a massive gluon partner in the minimal set of vector companions [\(\Delta R=0.2\) of a reconstructed electron are removed. If the nearest surviving small-R jet is within $\Delta R=0.4$ of the electron, then the electron is discarded. To reject hadronic jet candidates produced by bremsstrahlung from very energetic muons, the jet is required to have at least three associated tracks if it lies within a cone of $\Delta R=0.2$ around a muon candidate. However, if a surviving jet is separated from the nearest muon with transverse momentum $p^{\mu }_{\mathrm {T}}$ by $\Delta R < 0.04+10\,\text {Ge}\text {V}/p^{\mu }_{\mathrm {T}}$ up to a maximum of 0.4, the small-R jet is kept and the muon is removed instead; this reduces the background contributions due to muons from hadron decays. No dedicated overlap-removal procedure between large-R and small-R jets is performed. As high-$p_{\text {T}}$ electrons could deposit significant amounts of energy in the calorimeter to form large-R jets, the electron energy is removed from any overlap** large-R jets before the jet momentum requirements are applied to avoid double counting the electrons as large-R jets. This approach has a 20% better signal efficiency compared to rejecting large-R jets that overlap with a reconstructed electron.

5 Analysis strategy

5.1 Event selection

In the signal region (SR), events were recorded using either a set of single-electron triggers or a set of single-muon triggers. The single-electron triggers imposed a $p_{\text {T}}$ threshold of 26 $\text {Ge}\text {V}$ (24 $\text {Ge}\text {V}$ in 2015) and isolation requirements, or a $p_{\text {T}}$ threshold of 60 $\text {Ge}\text {V}$ and no isolation requirements [100] was performed to obtain an unbiased evaluation of the classifier performance. The classifier produces an output score referring to the predicted probability that the event contains LQs, which is then used as the final discriminant to separate LQ signal events from the SM backgrounds.

A natural basis of kinematic observables can be created, utilizing Lorentz symmetry to reduce unnecessary duplication of observables, in the rest frames of intermediate particle states, conditioned on the hypotheses of LQ pair, dileptonic $t\bar{t}$ or $Z+\mathrm {jets}$ decay processes. A suite of such discriminating variables is constructed using the recursive jigsaw reconstruction technique [\(p_{\text {T}}\) sum of the two leptons, the two large-R jet masses, and the reconstructed LQ mass. Figure 2 shows the distributions of the dilepton invariant mass in the $Z+\mathrm {jets}$ CR of the muon channel, and the reconstructed W mass based on a dileptonic $t\bar{t}$ hypothesis in the $t\bar{t}$ CR of the electron channel. In general, the kinematic variables show good agreement between data and the background expectation in the CRs. A complete list of the input variables is provided in Table 3.

Fig. 2

Distributions of the reconstructed W mass associated with the leading lepton assuming a dileptonic hypothesis in the $t\bar{t}$ CR after the simultaneous background-only fit of the electron channel CRs (left), and the dilepton invariant mass $m_{\ell \ell }$ in the $Z+\mathrm {jets}$ CR after the simultaneous background-only fit of the muon channel CRs (right). The bottom panels show the ratio of data to expected background. The hatched band represents the total uncertainty. The blue triangles indicate points that are outside the vertical range

In order to maximize the sensitivity of the BDT over a wide mass range, and ensure a smooth interpolation of the signal efficiency between the mass points where it was trained, a parameterized machine-learning approach [\(<\,3\%\) on the signal yield.

Other detector-related uncertainties come from uncertainties in the large-R jet mass scales and resolutions; lepton isolation and reconstruction; lepton trigger efficiencies, energy scales, and resolutions; the $E_{\text {T}}^{\text {miss}}$ reconstruction; pile-up modelling; and the jet-vertex-tagger requirement. Uncertainties in the object momenta are propagated to the $E_{\text {T}}^{\text {miss}}$ measurement, and additional uncertainties in $E_{\text {T}}^{\text {miss}}$ arising from the ‘soft’ energy are also considered. These all have negligible impact on the fitted signal yield (<3% each).

Table 2 Event yields in the signal and control regions before and after the background-only fit to data in the electron and muon channel. The quoted uncertainties include statistical and systematic uncertainties; for the $t\bar{t}$ and $Z+\mathrm {jets}$ backgrounds no cross-section uncertainty is included since it is a free parameter of the fit. The contributions from single top, $t\bar{t} V$, diboson and $W+\mathrm {jets}$ production are included in the ‘Others’ category. In the post-fit case, the uncertainties in the individual background components can be larger than the uncertainty in the sum of the backgrounds, due to the correlations between the fit parameters. Both signal models correspond to $m_{\text {LQ}} =$ 1500 $\text {Ge}\text {V}$ assuming 100% branching ratio into a hadronically decaying top quark and a charged lepton

6.3 Generator modelling uncertainties

Modelling uncertainties are estimated for the signal as well as Z+jets, $t\bar{t}$ and single-top-quark backgrounds. The modelling uncertainties are estimated by comparing simulated samples generated with different configurations, described in Sect. 3.

For the LQ signal, in addition to the cross-section uncertainties, the impact on the acceptance due to variations of the QCD scales, PDF and shower parameters was studied. These uncertainties were estimated from the envelope of independent pairs of renormalization and factorization scale variations by a factor of 0.5 and 2, by propagating the PDF and $\alpha _{\text {S}}$ uncertainties following the PDF4LHC15 prescription, and by considering two alternative samples generated with settings that increase or decrease the amount of QCD radiation. Both the PDF and scale variations have an impact below 15% for all bins considered, while variations of the underlying-event modelling have only a 1–2% effect.

For the Z+jets backgrounds, scale, PDF and $\alpha _{\text {S}}$ variations are considered and their effects are evaluated within the Sherpa event generator. Seven variations are considered for the renormalization and factorization scales, with the maximum shift within the envelope of those variations taken to estimate the effect of the scale uncertainty. The PDF variations include the variation of the nominal NNPDF3.0nnlo PDF as well as the central values of two other PDF sets, MMHT2014nnlo68cl [Pythia and Herwig, while kee** the same hard-scatter matrix-element calculation. The effects of extra initial- and final-state gluon radiation are estimated by comparing simulated samples generated with enhanced or reduced initial-state radiation, doubling the $h_\text {damp}$ parameter, and using different values of the radiation parameters [57]. The PDF uncertainty is estimated from the PDF4LHC15 error set. The dominant effect is from the final-state radiation estimation uncertainty and is about 6% of the signal yield.

In this analysis, the single-top-quark background comes mainly from the Wt-channel and is a minor background. Similarly to $t\bar{t}$ , uncertainties in the hard-scatter generation, the fragmentation and hadronization, the amount of additional radiation, and the PDF are considered. In addition, the uncertainty due to the treatment of the overlap between Wt-channel single top quark production and $t\bar{t}$ production is considered by comparing samples using the DS and DR methods (see Sect. 3). The dominant effect is from the uncertainty in the fragmentation and hadronization and is about 7% of the signal yield.

7 Statistical interpretation

The binned distributions of the BDT score in the SR and the overall number of events in the $t\bar{t}$ and $Z+\mathrm {jets}$ CR are used to test for the presence of a signal. Hypothesis testing is performed using a modified frequentist method as implemented in RooStats [112, 113] and is based on a profile likelihood that takes into account the systematic uncertainties as nuisance parameters that are fitted to the data. A simultaneous fit is performed in the SR and the two CRs, but done separately for the electron and muon channel. As the $t\bar{t}$ CR is built requiring an electron and muon, the same events are considered in the independent electron and muon channel fits.

The statistical analysis is based on a binned likelihood function $\mathcal {L}(\mu ,\theta )$ constructed as a product of Poisson probability terms over all bins considered in the search. This function depends on the signal strength parameter $\mu$, a multiplicative factor applied to the theoretical signal production cross section, and $\theta$, a set of nuisance parameters that encode the effect of systematic uncertainties in the signal and background expectations and are implemented in the likelihood function as Gaussian and log-normal constraints. Uncertainties in each bin due to the finite size of the simulated samples are also taken into account via dedicated constrained fit parameters. There are enough events in the CRs and the lowest BDT bin in the SR, where the signal contribution is small, to obtain a data-driven estimate of the $t\bar{t}$ and $Z+\mathrm {jets}$ normalizations and hence the normalizations of those two backgrounds are included as unconstrained nuisance parameters, $\mu _{t\bar{t}}$ and $\mu _{Z}$. Nuisance parameters representing systematic uncertainties are only included in the likelihood if either of the following conditions are met: the overall impact on the normalization in a given region is larger than 3%, or any single bin within the region has at least a 3% uncertainty. This is done separately for each region and for each template (signal or background). When the bin-by-bin statistical variation of a given uncertainty is significant, a smoothing algorithm is applied.

Fig. 4

Fit results (background-only) for the binned BDT output score distribution in the signal region of the electron (left) and muon (right) channel, and the overall number of events in the $t\bar{t}$ and $Z+\mathrm {jets}$ control regions. The lower panel shows the ratio of data to the fitted background yields. The band represents the systematic uncertainty after the maximum-likelihood fit

The test statistic $q_{\mu }$ is defined as the profile likelihood ratio, $q_{\mu }=-2\mathrm {ln}(\mathcal {L}(\mu ,\hat{\hat{\theta }}_{\mu })/ \mathcal {L}(\hat{\mu }, \hat{\theta }))$, where $\hat{\mu }$ and $\hat{\theta }$ are the values of the parameters that maximize the likelihood function, and $\hat{\hat{\theta }}_{\mu }$ are the values of the nuisance parameters that maximize the likelihood function for a given value of $\mu$. The compatibility of the observed data with the background-only hypothesis is tested by setting $\mu =0$ in the profile likelihood ratio: $q_{0}=-2\mathrm {ln}(\mathcal {L}(0,{\hat{\hat{\theta }}}_{0})/\mathcal {L}({\hat{\mu }},{\hat{\theta }}))$. Upper limits on the signal production cross section for each of the signal scenarios considered are derived by using $q_{\mu }$ in the so-called CL$_{\mathrm {s}}$ method [114, 115]. For a given signal scenario, values of the production cross section (parameterized by $\mu$) yielding CL$_{\mathrm {s}}<0.05$, where CL$_{\mathrm {s}}$ is computed using the asymptotic approximation [116], are excluded at $\ge 95$% confidence level (CL).

8 Results

8.1 Likelihood fit results

Fig. 5

Upper limits at 95% CL on the cross section of LQ pair production as a function of LQ mass, assuming a branching ratio $\mathcal {B}$ $(\mathrm {LQ} \rightarrow t\ell ^{\pm }) = 1$, for the electron (left) and muon (right) channel. Observed limits are shown as a black solid line and expected limits as a black dashed line. The green and yellow shaded bands correspond to ±1 and ±2 standard deviations, respectively, around the expected limit. The red curve and band show the nominal theoretical prediction and its ±1 standard deviation uncertainty

Fig. 6

Lower exclusion limits on the leptoquark mass for scalar leptoquark pair production as a function of the branching ratio into a top quark and an electron (left) or a muon (right) at 95% CL. The observed nominal limits are indicated by a black solid curve, with the surrounding red dotted lines obtained by varying the signal cross section by uncertainties from PDFs, renormalization and factorization scales, and the strong coupling constant $\alpha _{\text {S}}$. Expected limits are indicated with a black dashed curve, with the yellow and green bands indicating the ±1 standard deviation and ±2 standard deviation excursions due to experimental and modelling uncertainties

The expected and observed event yields in the signal and control regions before and after fitting the background-only hypothesis to data, including all uncertainties, are listed in Table 2. The total uncertainty shown in the table is the uncertainty obtained from the full fit, and is therefore not identical to the sum in quadrature of each component, due to the correlations between the fit parameters. A comparison of the post-fit agreement between data and prediction for the signal and control regions is shown in Fig. 4. In the electron (muon) channel, the ratio of the $t\bar{t}$ total post-fit yield over the pre-fit yield is $0.90\pm 0.25$ ($0.84\pm 0.24$). The ratio of the $Z+\mathrm {jets}$ total post-fit yield over the pre-fit yield is $0.95\pm 0.20$ ($0.87\pm 0.10$). None of the individual uncertainties are significantly constrained by data.

The probability that the data is compatible with the background-only hypothesis is estimated by integrating the distribution of the test statistic, approximated using the asymptotic formulae, above the observed value of $q_{0}$.² This value is computed for each signal scenario considered, defined by the assumed mass of the leptoquark. The lowest local p-value is found to be $\sim$11% (10%), for a LQ mass of 1450 (1600) $\text {Ge}\text {V}$ in the electron (muon) channel. Thus no significant excess above the background expectation is found.

8.2 Limits on LQ pair production

Upper limits at the 95% CL on the LQ pair-production cross section, for an assumed value of $\beta =1$, are set as a function of the LQ mass $m_{\mathrm {LQ}}$ and compared with the theoretical prediction (Fig. 5). The resulting lower limit on $m_{\mathrm {LQ}}$ is determined using the central value of the theoretical NNLO+NNLL cross-section prediction. The observed (expected) lower limits on $m_{\mathrm {LQ}}$ are found to be 1480 (1560) $\text {Ge}\text {V}$ and 1470 (1540) $\text {Ge}\text {V}$ for the electron and muon channel respectively. The sensitivity of the analysis is limited by the statistical uncertainty of the data. Including all systematic uncertainties degrades the expected mass limits by only around 10 $\text {Ge}\text {V}$, and for a mass of 1.5 $\text {Te}\text {V}$ the cross-section limits increase by less than 7% in both the electron and muon channel.

Exclusion limits on LQ pair production are also obtained for different values of $m_{\mathrm {LQ}}$ as a function of the branching ratio ($\mathcal {B}$) into a charged lepton and a top quark (Fig. 6). The theoretical cross section was scaled by the branching ratio, and then used to obtain the corresponding limit. The full statistical interpretation is performed for each 0.1 step in $\mathcal {B}$, covering the full plane.

9 Conclusion

A search for pair production of scalar leptoquarks, each decaying into a top quark and either an electron or a muon has been presented, targeting the high-mass region in which the decay products of each top quark are contained within a single large-radius jet. The analysis is based on tight selection criteria to reduce the SM backgrounds. The normalizations of the dominant $Z+\mathrm {jets}$ and $t\bar{t}$ backgrounds were determined simultaneously in a profile likelihood fit to the binned output score of a boosted decision tree in the signal region and two dedicated control regions. The data used in this search correspond to an integrated luminosity of 139 $\mathrm {fb}^{-1}$ of pp collisions with a centre-of-mass energy $\sqrt{s}=13~\text {Te}\text {V}$ recorded by the ATLAS experiment in the whole of Run 2 of the LHC. The observed data distributions are compatible with the expected Standard Model background and no significant excess is observed. Lower limits on the leptoquark masses are set at 1480 $\text {Ge}\text {V}$ and 1470 $\text {Ge}\text {V}$ for the electron and muon channel, respectively.

Appendix

Table 3 The discriminating variables used in the signal–background discrimination training can be classified into five different groups. The first three groups include kinematic variables that are physics-based rather than detector-based, conditioned on different physics process hypotheses: LQ, dileptonic $t\bar{t}$ and leptonic Z decay hypothesis. These physics-based kinematic variables include the invariant masses and the momenta of intermediate and final-state particles in their parent’s rest frame. In the dileptonic $t\bar{t}$ decay hypothesis, the combinatoric ambiguity in the small-R-jet–lepton pairing is resolved using the ‘min $\Delta M_{\mathrm {top}}$ approach’ of the recursive jigsaw reconstruction technique [101]. The reconstructed hemisphere of the decay process associated with the leading (subleading) lepton is labelled with 1 (2). The fourth group of variables is detector-based and defined in the lab frame. These variables are related to the event-level activity of visible objects or missing transverse momentum. The last group of variables is used to identify the three-prong jet structure of hadronic top-quark decays and is used only in the muon channel