1 Introduction

With the development of nonlinear science, nonlinear evolution equations (NLEEs) have been applied to many nonlinear complex systems in physics and engineering science. Especially, the famous nonlinear Schrödinger (NLS) equation has attracted much attention due to its important applications in nonlinear optics and plasma physics. In this paper, we consider a cubic and quintic NLS equation, that is the Eckhaus–Kundu (EK) equation proposed by Kundu [1], Calogero and Eckhaus [2] successively

$$\begin{aligned} i q_{t}+q_{x x}-2 \sigma |q |^{2} q+\delta ^{2}|q |^{4} q+2 i \delta \left( |q |^{2}\right) _{x} q=0, \quad x \in {\mathbb {R}}, \end{aligned}$$
(1)

where q(x, t) is a complex function of spatial variable x and time variable t, \(\sigma \) and \(\delta \) are real constants. The third and fourth terms of equation (1) represent nonlinear dispersion effect and non-Kerr nonlinear effect respectively, the last term is a nonlinear term that results from the time-retarded induced Raman process. Equation (1) is a basic model to simulate the propagation of optical solitons in Kerr medium, especially in quantum field theory to describe the propagation of ultrashort femtosecond pulses in optical fibers [3]. In addition, due to the re-scaling caused by the high-order nonlinear terms of equation (1), [4] examined the fine detail of Benjamin’s side-band instability. These results can be used to investigate the change in form of the soliton solution near the stability boundary, which naturally leads to the possibilities both of more general ‘soliton’ solutions and the application of inverse scattering methods. In the field of nonlinear optics, [5] discussed some properties and soliton solutions of equation (1), and how the perturbations of the equation affect the integrability of the equation, and they also considered a feasibility of a long-distance-high-bit-rate optical transmission system by use of solitons.

In [6,7,8,9,10,11], Darboux transformation method, Hirota bilinear method and Riemann-Hilbert method et al. were used to obtain the bright, dark soliton solutions etc. and study the long-time asymptotic behavior of N-soliton solutions of equation (1), respectively. In the recent 2 years, the research on equation (1) is still hot. In [12,13,14,15], N-soliton solutions and new soliton solutions of equation (1) were constructed by Hirota bilinear method, \(\lambda \)-symmetry method and \({\bar{\partial }}\)-dressing method et al.

It is of great practical significance to study the stability of solution for a nonlinear system. The orbital stability theory proposed by Benjamin [16] in 1972 belongs to the category of nonlinear stability, later on, Bona [17, 18], Weinstein [19, 20], Grillakis, Shatah and Strauss [21, 22] further improved the theory. In terms of the solitary wave solution to EK equation (1), we study its orbital stability in this paper. Since equation (1) is difficult to be expressed as a standard Hamiltonian form, the Grillakis–Shatah–Strauss theory [21, 22] about the orbital stability of soliton solution for nonlinear Hamiltonian system cannot be directly applied. In order to overcome this difficulty, inspired by the “stability theorem” of the introduction part and the derivation process in [22], we solve this problem by constructing three new conserved quantities. Since the high-order nonlinear terms of the equation studied in this paper have both cubic and quintic terms, the construction of conserved quantities E(q), \(Q_{1}(q)\), \(Q_{2}(q)\) and operator \(H_{\omega ,v}\) is more complicated, which makes the spectral analysis of operator \(H_{\omega ,v}\) more difficult, by using special techniques and detailed spectral analysis, then we obtain the conclusion that the soliton solution of EK equation (1) is orbitally stable under the condition of \(\sigma <0\), \(4 \omega + v^{2}<0\).

This paper is organized as follows. In Sect. 2, we give the solitary wave solution of equation (1) by undetermined coefficient method. In Sect. 3, we construct three new conserved quantities, through further analysis, we give the assumption and conditions that the orbital stability of the solitary wave solution for equation (1) should satisfy. In Sect. 4, we prove that the solitary wave of equation (1) is orbitally stable under the assumption and conditions given in Sect. 3 by special techniques and detailed spectral analysis.

2 Solitary Wave Solution of Eckhaus–Kundu Equation

Assume that equation (1) has bounded traveling wave solution in the form of

$$\begin{aligned} q(x, t)=e^{-i \omega t} e^{i \psi (x-v t)} a(x-v t)=e^{-i \omega t} e^{i \psi (\xi )} a(\xi ), \xi =x-v t, \end{aligned}$$
(2)

let

$$\begin{aligned} {\hat{a}}(\xi )={\hat{a}}(x-v t)=e^{i \psi (x-v t)} a(x-v t), \end{aligned}$$
(3)

then the traveling wave solution can be expressed as \(q=e^{-i \omega t} {\hat{a}}(x-v t)\). Substituting it into (1), we can obtain

$$\begin{aligned} -{\hat{a}}_{x x}-g(a^{2}) {\hat{a}}-2 i \delta \left( a^{2}\right) _{x} {\hat{a}}-\omega {\hat{a}}+i v {\hat{a}}_{x}=0, \end{aligned}$$
(4)

where the function \(g(\cdot )\) is defined as

$$\begin{aligned} g(z)=-2 \sigma z+\delta ^{2} z^{2}, \quad \forall z \in {\mathbb {R}}. \end{aligned}$$
(5)

Substitute (3) into (4) and let the real and imaginary parts in (4) be equal to zero, we have

$$\begin{aligned}{} & {} a^{\prime \prime }+\left[ g(a^{2})-\left( \psi ^{\prime }\right) ^{2}+\omega +v \psi ^{\prime }\right] a=0, \end{aligned}$$
(6)
$$\begin{aligned}{} & {} \psi ^{\prime \prime } a+2 \psi ^{\prime } a^{\prime }+4 \delta a^{2} a^{\prime }-v a^{\prime }=0. \end{aligned}$$
(7)

Multiplying equation (7) with an integrating factor a, then (7) can be expressed as

$$\begin{aligned} \left( a^{2} \psi ^{\prime }\right) ^{\prime }+\left( \delta a^{4}-\frac{v}{2} a^{2}\right) ^{\prime }=0, \end{aligned}$$
(8)

after integration, we have

$$\begin{aligned} \psi ^{\prime }(\xi )=\frac{v}{2}-\delta a^{2}(\xi ), \end{aligned}$$
(9)

then (6) can be transformed to

$$\begin{aligned} a^{\prime \prime }(\xi )-2\sigma \left( a^{3}(\xi )+m a(\xi )\right) =0, \end{aligned}$$
(10)

where \(m=\frac{4 \omega +v^{2}}{-8 \sigma }\).

Make a transformation \(a(\xi )=\sqrt{\varphi (\xi )}\), from (10) we have

$$\begin{aligned} -\left( \varphi ^{\prime }\right) ^{2}+2 \varphi \varphi ^{\prime \prime }-8 \sigma \left( \varphi ^{3}+m \varphi ^{2}\right) =0. \end{aligned}$$
(11)

According to the undetermined coefficient method, we assume that equation (11) has the solution in the form of

$$\begin{aligned} \varphi (\xi )=\frac{A \cdot {\text {sech}}^{2}\left( C\left( \xi -\xi _{0}\right) \right) }{2+B \cdot {\text {sech}}^{2}\left( C\left( \xi -\xi _{0}\right) \right) }, \end{aligned}$$
(12)

where A, B and C are constants to be determined. Substituting (12) into (11), we can obtain

$$\begin{aligned} A=-4 m, \quad B=0, \quad C=\sqrt{2 \sigma m}, \quad \sigma m>0, \end{aligned}$$

then equation (11) has the following solution in the form of

$$\begin{aligned} \varphi (\xi )=-2 m {\text {sech}}^{2}\left( \sqrt{2 \sigma m}\left( \xi -\xi _{0}\right) \right) , \quad \sigma m>0. \end{aligned}$$
(13)

It’s easy to verify when \(m<0\), \(\varphi (\xi )>0\). Now let \(l=2 \sigma m=-\frac{4 \omega +v^{2}}{4}\), and note that \(a(\xi )=\sqrt{\varphi (\xi )}\), based on the above constructivity method of mathematics, the following theorem can be obtained:

Theorem 2.1

If \(\sigma <0\) and \(4 \omega + v^{2}<0\), equation (10) has the bounded analytic solution

$$\begin{aligned} a^{\pm }(\xi )=\pm \sqrt{-\frac{l}{\sigma }} {\text {sech}}\left( \sqrt{l}\left( \xi -\xi _{0}\right) \right) , \end{aligned}$$
(14)

where \(l=-\omega -\frac{v^{2}}{4}\). Further equation (1) has solitary wave solution \(q(x, t)=e^{-i \omega t} e^{i \psi (x-v t)} a^{\pm }(x-v t)\), where \(a(\xi )\) is given by (14), \(\psi (\xi )\) satisfies \(\psi ^{\prime }(\xi )=\frac{v}{2}-\delta \left( a^{\pm }(\xi )\right) ^{2}\).

3 Orbital Stability of Solitary Wave Solution for Eckhaus–Kundu Equation

In this section, we only consider the case of \(a^{+}(\xi )>0\) in (14) (the discussion on the case of taking the negative sign is similar). Therefore, it is always assumed that \(a(\xi )>0\).

The initial value problem of equation (1) is considered in function space \(X=H^{1}({\mathbb {R}})\) as follows

$$\begin{aligned}{} & {} q_{t}=i q_{x x}-2 i \sigma |q |^{2} q+i \delta ^{2}|q |^{4} q-2 \delta \left( |q |^{2}\right) _{x} q, \end{aligned}$$
(15)
$$\begin{aligned}{} & {} q(0, x)=q_{0}(x), \quad x \in {\mathbb {R}}, \end{aligned}$$
(16)

where the complex space \(X=H^{1}({\mathbb {R}})\) has the following real inner product

$$\begin{aligned} (f, g)={\text {Re}} \int \limits _{{\mathbb {R}}}\left( f_{x} {\bar{g}}_{x}+f {\bar{g}}\right) d x, \quad \forall f, g \in X, \end{aligned}$$
(17)

and \(X^{*}=H^{-1}({\mathbb {R}})\) expresses the dual space of X, we define the natural isomorphism \(I:X \rightarrow X^{*}\) as

$$\begin{aligned} \langle If, g\rangle =(f, g), \end{aligned}$$
(18)

where \(\langle \cdot , \cdot \rangle \) represents the pairing of X and \(X^{*}\), i.e.

$$\begin{aligned} \langle u, f\rangle ={\text {Re}} \int \limits _{{\mathbb {R}}} u {\bar{f}} d x, \end{aligned}$$
(19)

from (17)–(19), we have

$$\begin{aligned} I=-\frac{\partial ^{2}}{\partial x^{2}}+1. \end{aligned}$$
(20)

Let \(T_{1}\) and \(T_{2}\) be the unitary operator groups with single parameter in space X, defined as

$$\begin{aligned}{} & {} T_{1}\left( s_{1}\right) \phi (\cdot )=e^{-s_{1} i} \phi (\cdot ), \quad \forall \phi (\cdot ) \in X, s_{1} \in {\mathbb {R}}, \end{aligned}$$
(21)
$$\begin{aligned}{} & {} T_{2}\left( s_{2}\right) \phi (\cdot )=\phi \left( \cdot -s_{2}\right) , \quad \forall \phi (\cdot ) \in X, s_{2} \in {\mathbb {R}}, \end{aligned}$$
(22)

then we can get \(T_{1}^{\prime }(0)=-i, T_{2}^{\prime }(0)=-\frac{\partial }{\partial x}\).

Taking the higher-order nonlinear terms of the equation studied in [23] as cubic and quintic, the existence theorem of the solution to the initial value problem of equation (1) can be obtained as follows:

Theorem 3.1

For any \(q_{0} \in X=H^{1}({\mathbb {R}})\), the initial value problem (15)–(16) of equation (1) has the unique solution \(q \in C\left( \left[ 0, T_{\text{ max } }\right] ; H^{1}({\mathbb {R}})\right) \), which satisfies \(q(0)=q_{0}\).

From the definitions of \(T_{1}\) and \(T_{2}\), the solitary wave solution of equation (15) in Theorem 2.1 can be written as \(T_{1}(\omega t) T_{2}(v t) {\hat{a}}_{\omega , v}(x)\), where \({\hat{a}}_{\omega , v}(x)\) is determined by (3) and (14). Therefore, in terms of the solitary wave \(T_{1}(\omega t) T_{2}(v t) {\hat{a}}_{\omega , v}(x)\), we principally study its orbital stability in the following discussion. Noting that equation (15) has the symmetries of phase and translation, then the orbital stability is defined as follows.

Definition 3.1

The solitary wave \(T_{1}(\omega t) T_{2}(v t) {\hat{a}}_{\omega , v}(x)\) is orbitally stable: for \(\forall \varepsilon >0\), there exists \(\tau >0\), if \(\left\| q_{0}-{\hat{a}}_{\omega , v}\right\| _{X}<\tau \), q(t) is the solution of equation on \([0,t_0)\), \(q(0)=q_{0}\), then q(t) can be continuously extended to the solution on \(0 \le t<\infty \), and

$$\begin{aligned} \sup _{0<t<\infty } \inf _{s_{1} \in {\mathbb {R}}} \inf _{s_{2} \in {\mathbb {R}}}\left\| q(t)-T_{1}\left( s_{1}\right) T_{2}\left( s_{2}\right) {\hat{a}}_{\omega , v}\right\| _{X}<\varepsilon . \end{aligned}$$

Otherwise, it is called that \(T_{1}(\omega t) T_{2}(v t) {\hat{a}}_{\omega , v}(x)\) is orbitally unstable.

Since equation (15) studied in this paper is difficult to be transformed into a standard Hamiltonian system, we cannot directly apply the Grillakis–Shatah–Strauss theory [21, 22] about the orbital stability of soliton solution for nonlinear Hamiltonian system to equation (15). However, based on the “stability theorem” of introduction part in [22], Assumption 1 to Assumption 3 that make the theorem hold and the derivation process in Sects. 3 and 4 of [22], for the initial value problem of the equation, it can be seen that when there exists the local solution, as long as the appropriate conserved quantities E(q) and \(Q_{\sigma }(q)\) are found to hold the assumptions 2 and 3, then even if the equation can’t be transformed to a standard Hamiltonian system, we can still obtain the result that the solitary wave is orbitally stable. Based on the aforesaid analysis, three new conserved quantities are constructed as follows

$$\begin{aligned}{} & {} E(q)=\frac{1}{2} \int \limits _{{\mathbb {R}}}\left\{ |\left( \partial _{x}+i \delta |q |^{2}\right) q|^{2}+\sigma |q |^{4}\right\} d x, \end{aligned}$$
(23)
$$\begin{aligned}{} & {} Q_{1}(q)=\frac{1}{2} \int \limits _{{\mathbb {R}}}|q |^{2} d x, \end{aligned}$$
(24)
$$\begin{aligned}{} & {} Q_{2}(q)=\int \limits _{{\mathbb {R}}}\left\{ \frac{1}{2} {\text {Im}}\left( {\bar{q}} q_{x}\right) +\frac{\delta }{2}|q |^{4}\right\} d x. \end{aligned}$$
(25)

It’s easy to verify that E(q), \(Q_{1}(q)\) and \(Q_{2}(q)\) are \(C^{2}\) functionals described on X, and the corresponding first-order Fréchet derivatives are denoted as \(\left\langle E^{\prime }(q), v\right\rangle ,\left\langle Q_{1}^{\prime }(q), v\right\rangle \) and \(\left\langle Q_{2}^{\prime }(q), v\right\rangle \), the corresponding second-order derivatives are denoted as \(\left\langle E^{\prime \prime }(q) \omega , v\right\rangle ,\left\langle Q_{1}^{\prime \prime }(q) \omega , v\right\rangle \) and \(\left\langle Q_{2}^{\prime \prime }(q) \omega , v\right\rangle \), where \(E^{\prime }, Q_{1}^{\prime }, Q_{2}^{\prime }: X \rightarrow X^{*}\).

By computing, we can get

$$\begin{aligned} E^{\prime }(q)=-q_{x x}+4 \delta ^{2}|q |^{4} q-g(|q |^{2}) q-4 i \delta |q |^{2} q_{x},\\ Q_{1}^{\prime }(q)=q,\\ Q_{2}^{\prime }(q)=-i q_{x}+2 \delta |q |^{2} q. \end{aligned}$$

It can be proved that E(q), \(Q_{1}(q)\) and \(Q_{2}(q)\) are invariant under the effect of \(T_{1}\) and \(T_{2}\).

Proposition 3.1

For arbitrary \(s_{1}, s_{2}\in {\mathbb {R}}\), we can get the following equalities:

$$\begin{aligned} \left\{ \begin{aligned} E\left( T_{1}\left( s_{1}\right) T_{2}\left( s_{2}\right) q\right)&=E(q),\\ Q_{1}\left( T_{1}\left( s_{1}\right) T_{2}\left( s_{2}\right) q\right)&=Q_{1}(q), \\ Q_{2}\left( T_{1}\left( s_{1}\right) T_{2}\left( s_{2}\right) q\right)&=Q_{2}(q), \end{aligned}\right. \end{aligned}$$
(26)

and for \(\forall t \in {\mathbb {R}}\), q(t) is a flow of (15), we have

$$\begin{aligned} E(q(t))=E(q(0)),\quad Q_{1}(q(t))=Q_{1}(q(0)),\quad Q_{2}(q(t))=Q_{2}(q(0)). \end{aligned}$$
(27)

Proof

First, we prove the conservation of E(q). We define

$$\begin{aligned} H(q):=-q_{x x}+2 \sigma |q |^{2} q-\delta ^{2}|q |^{4} q-2 i\left( |q |^{2}\right) _{x} q, \end{aligned}$$
(28)

from (15), we have

$$\begin{aligned} q_{t}=-i H(q). \end{aligned}$$

Since

$$\begin{aligned} \begin{aligned} -2 i\left( |q |^{2}\right) _{x} q&=-2 i\left\{ 2|q |^{2} q_{x}+\left( q {\bar{q}}_{x}-{\bar{q}} q_{x}\right) q\right\} \\&=-4 i|q |^{2} q_{x}-4 p(q) q, \end{aligned} \end{aligned}$$

where \(p(q)=\frac{{\bar{q}} q_{x}-q {\bar{q}}_{x}}{2 i}={\text {Im}}\left( {\bar{q}} q_{x}\right) \), combined with the expression of \(E^{\prime }(q)\), we can get

$$\begin{aligned} H=E^{\prime }-4 \delta q\left( p(q)+\delta |q |^{4}\right) . \end{aligned}$$
(29)

On the other hand, it is relatively straightforward to derive the conservation law

$$\begin{aligned} \partial _{t}\left( \frac{|q |^{2}}{2}\right) +\partial _{x}\left( p+\delta |q |^{4}\right) =0. \end{aligned}$$
(30)

Since

$$\begin{aligned} \left\langle H, q_{t}\right\rangle =\langle H,-i H\rangle =0, \end{aligned}$$

then the time derivative of the energy is

$$\begin{aligned} \frac{d E(q)}{d t}=\left\langle E^{\prime }, q_{t}\right\rangle =\left\langle E^{\prime }-H, q_{t}\right\rangle +\left\langle H, q_{t}\right\rangle =\left\langle E^{\prime }-H, q_{t}\right\rangle , \end{aligned}$$

and subsequently, on account of (30), we get

$$\begin{aligned} \begin{aligned} \left\langle E^{\prime }-H, q_{t}\right\rangle =&4 \delta \int \limits _{{\mathbb {R}}} \partial _{t}\left( \frac{|q |^{2}}{2}\right) \left( p(q)+\delta |q |^{4}\right) d x \\ =&-2 \delta \int \limits _{{\mathbb {R}}} \partial _{x}\left\{ \left( p(q)+\delta |q |^{4}\right) ^{2}\right\} d x=0, \end{aligned} \end{aligned}$$
(31)

then we obtain

$$\begin{aligned} \frac{d E(q)}{d t}=0. \end{aligned}$$

Now we prove the conservation of \(Q_{1}(q)\). According to (30), we can compute

$$\begin{aligned} \begin{aligned} \frac{d Q_{1}(q)}{d t}&=\left\langle Q_{1}^{\prime }, q_{t}\right\rangle =\left\langle q, q_{t}\right\rangle \\&=\int \limits _{{\mathbb {R}}} \partial _{t}\left( \frac{|q |^{2}}{2}\right) d x=\int \limits _{{\mathbb {R}}} \partial _{x}\left( p+\delta |q |^{4}\right) d x=0. \end{aligned} \end{aligned}$$

Finally, we prove the conservation of \(Q_{2}(q)\). Since

$$\begin{aligned} \left( q_{x}+2 i \delta |q |^{2} q\right) _{x}=q_{x x}+2 i \delta \left( |q |^{2}\right) _{x} q+2 i \delta |q |^{2} q_{x}, \end{aligned}$$

here H can be expressed as

$$\begin{aligned} H=-\left( q_{x}+2 i \delta |q |^{2} q\right) _{x}+2 \sigma |q |^{2} q-\delta ^{2}|q |^{4} q+2 i \delta |q |^{2} q_{x}. \end{aligned}$$

Then, we can compute

$$\begin{aligned} \begin{aligned} \frac{d Q_{2}(q)}{d t}=&\left\langle Q_{2}^{\prime }, q_{t}\right\rangle =\left\langle -i\left( q_{x}+2 i \delta |q |^{2} q\right) ,-i H\right\rangle =\left\langle q_{x}+2 i \delta |q |^{2} q, H\right\rangle \\ =&\left\langle q_{x}+2 i \delta |q |^{2} q,-\left( q_{x}+2 i \delta |q |^{2} q\right) _{x}+2 \sigma |q |^{2} q-\delta ^{2}|q |^{4} q+2 i \delta |q |^{2} q_{x}\right\rangle \\ =&-\frac{1}{2} \int \limits _{{\mathbb {R}}} \partial _{x}\left( |q_{x}+2 i \delta |q |^{2} q|^{2}\right) d x+\frac{2}{3} \delta ^{2} \int \limits _{{\mathbb {R}}} \partial _{x}\left( |q |^{6}\right) d x \\&+\left\langle q_{x}, 2 \sigma |q |^{2} q-\delta ^{2}|q |^{4} q+2 i \delta |q |^{2} q_{x}\right\rangle \\ =&-\frac{1}{2} \int \limits _{{\mathbb {R}}} \partial _{x}\left( |q_{x}+2 i \delta |q |^{2} q|^{2}\right) d x+\frac{1}{2} \delta ^{2} \int \limits _{{\mathbb {R}}} \partial _{x}\left( |q |^{6}\right) d x+\frac{\sigma }{2} \int \limits _{{\mathbb {R}}} \partial _{x}\left( |q |^{4}\right) d x \\ =&0. \end{aligned} \end{aligned}$$

From the above, we obtain \(\frac{d E(q)}{d t}=\frac{d Q_{1}(q)}{d t}=\frac{d Q_{2}(q)}{d t}=0\), which means that for \(\forall t \in {\mathbb {R}}\), we have \(E(q(t))=E(q(0)), Q_{1}(q(t))=Q_{1}(q(0)), Q_{2}(q(t))=Q_{2}(q(0))\). \(\square \)

For \({\hat{a}}\), we can also verify the following equation holds:

$$\begin{aligned} E^{\prime }({\hat{a}}(x))-\omega Q_{1}^{\prime }({\hat{a}}(x))-v Q_{2}^{\prime }({\hat{a}}(x))=0. \end{aligned}$$
(32)

Actually, from (3), (4) and (8) we have

$$\begin{aligned} \begin{array}{l} E^{\prime }({\hat{a}}(x))-\omega Q_{1}^{\prime }({\hat{a}}(x))-v Q_{2}^{\prime }({\hat{a}}(x)) \\ =-{\hat{a}}_{x x}+4 \delta ^{2} a^{4} {\hat{a}}-g(a^{2}) {\hat{a}}-4 i \delta a^{2} {\hat{a}}_{x}-\omega {\hat{a}}+i v {\hat{a}}_{x}-2 \delta v a^{2} {\hat{a}} \\ =4 \delta ^{2} a^{4} {\hat{a}}-2 \delta v a^{2} {\hat{a}}+4 \delta i a {\hat{a}} a_{x}-4 i \delta a^{2} {\hat{a}}_{x} \\ =e^{i \psi }\left[ 4 \delta ^{2} a^{5}-2 \delta v a^{3}+4 i \delta a^{2} a_{x}-4 i \delta a^{2}\left( i \psi ^{\prime } a+a^{\prime }\right) \right] =0. \end{array} \end{aligned}$$

For \(\forall q, \phi \in X\), by computing, we can get

$$\begin{aligned} \begin{aligned} E^{\prime \prime }(q) \phi =&-\phi _{x x}+4 \delta ^{2}\left( 3|q |^{4} \phi +2|q |^{2} q^{2} {\bar{\phi }}\right) -g(|q |^{2}) \phi \\&-g^{\prime }(|q |^{2})({\bar{q}} \phi +q {\bar{\phi }}) q-4 i \delta \left( |q |^{2} \phi _{x}+\phi {\bar{q}} q_{x}+{\bar{\phi }} q q_{x}\right) , \\ Q_{1}^{\prime \prime }(q) \phi =&\phi , \\ Q_{2}^{\prime \prime }(q) \phi =&-i \phi _{x}+4 \delta |q |^{2} \phi +2 \delta q^{2} {\bar{\phi }}. \end{aligned} \end{aligned}$$

The operator \(H_{\omega , v}:=X \rightarrow X^{*}\) is defined as follows

$$\begin{aligned} H_{\omega , v}=E^{\prime \prime }({\hat{a}})-\omega Q_{1}^{\prime \prime }({\hat{a}})-v Q_{2}^{\prime \prime }({\hat{a}}), \end{aligned}$$
(33)

thus the following equality can be given for \(\forall \phi \in X\),

$$\begin{aligned} \begin{aligned} H_{\omega , v} \phi =&-\phi _{x x}+4 \delta ^{2}\left( 3 a^{4} \phi +2 a^{2} {\hat{a}}^{2} {\bar{\phi }}\right) -g(a^{2}) \phi \\&-g^{\prime }(a^{2})(\overline{{\hat{a}}} \phi +{\hat{a}} {\bar{\phi }}) {\hat{a}}-4 i \delta \left( a^{2} \phi _{x}+\phi \overline{{\hat{a}}} {\hat{a}}_{x}+{\bar{\phi }} {\hat{a}} {\hat{a}}_{x}\right) \\&-\omega \phi +i v \phi _{x}-4 \delta v a^{2} \phi -2 \delta v {\hat{a}}^{2} {\bar{\phi }}. \end{aligned} \end{aligned}$$
(34)

It can be seen that \(H_{\omega , v}\) is self-adjoint, i.e. \(I^{-1} H_{\omega , v}\) is a bounded self-adjoint operator on X. The spectrums of \(H_{\omega , v}\) consist of real numbers \(\lambda \) which make \(H_{\omega , v}-\lambda {\mathbb {I}}\) non-invertible (here \({\mathbb {I}}\) is the identity operator).

Since \(T_{1}^{\prime }(0)=-i, T_{2}^{\prime }(0)=-\frac{\partial }{\partial x}\), we get

$$\begin{aligned} T_{1}^{\prime }(0) {\hat{a}}(x)=-i {\hat{a}}(x),\\ T_{2}^{\prime }(0) {\hat{a}}(x)=-{\hat{a}}_{x}(x). \end{aligned}$$

Therefore, we can obtain

$$\begin{aligned}{} & {} H_{\omega , v} T_{1}^{\prime }(0) {\hat{a}}(x)=0, \end{aligned}$$
(35)
$$\begin{aligned}{} & {} H_{\omega , v} T_{2}^{\prime }(0) {\hat{a}}(x)=0. \end{aligned}$$
(36)

In fact,

$$\begin{aligned}{} & {} \begin{aligned} H_{\omega , v} T_{1}^{\prime }(0) {\hat{a}}(x)=&H_{\omega , v}(-i {\hat{a}}(x)) \\ =&i {\hat{a}}_{x x}+4 \delta ^{2}\left( -3 i a^{4} {\hat{a}}+2 i a^{4} {\hat{a}}\right) +i g(a^{2}) {\hat{a}} \\&-g^{\prime }(a^{2})\left( i a^{2}-i a^{2}\right) {\hat{a}}-4 i \delta \left( i a^{2} {\hat{a}}_{x}-i a^{2} {\hat{a}}_{x}-i a^{2} {\hat{a}}_{x}\right) \\&+i \omega {\hat{a}}+v {\hat{a}}_{x}+4 i \delta v a^{2} {\hat{a}}-2 i \delta v a^{2} {\hat{a}} \\ =&-i\left[ E^{\prime }({\hat{a}})-\omega Q_{1}^{\prime }({\hat{a}})-v Q_{2}^{\prime }({\hat{a}})\right] =0, \end{aligned}\\{} & {} \begin{aligned} H_{\omega , v} T_{2}^{\prime }(0) {\hat{a}}(x)=&H_{\omega , v}\left( -{\hat{a}}_{x}(x)\right) \\ =&{\hat{a}}_{x x x}-4 \delta ^{2}\left( 3 a^{4} {\hat{a}}_{x}+2 a^{2} {\hat{a}}^{2} \overline{{\hat{a}}}_{x}\right) +g(|a |^{2}) {\hat{a}}_{x} \\&+g^{\prime }(a^{2})\left( \overline{{\hat{a}}} {\hat{a}}_{x}+{\hat{a}} \overline{{\hat{a}}}_{x}\right) {\hat{a}}+4 i \delta \left( a^{2} {\hat{a}}_{x x}+{\hat{a}}_{x} \overline{{\hat{a}}} {\hat{a}}_{x}+{\hat{a}} \overline{{\hat{a}}}_{x} {\hat{a}}_{x}\right) \\&+\omega {\hat{a}}_{x}-i v {\hat{a}}_{x x}+4 \delta v|a |^{2} {\hat{a}}_{x}+2 \delta v {\hat{a}}^{2} \overline{{\hat{a}}}_{x} \\ =&-\frac{\partial }{\partial x}\left[ E^{\prime }({\hat{a}})-\omega Q_{1}^{\prime }({\hat{a}})-v Q_{2}^{\prime }({\hat{a}})\right] =0. \end{aligned} \end{aligned}$$

Let \(Z=\left\{ k_{1} T_{1}^{\prime }(0) {\hat{a}}(x)+\right. \left. k_{2} T_{2}^{\prime }(0) {\hat{a}}(x) \mid k_{1}, k_{2} \in {\mathbb {R}}\right\} \), it can be seen from (35) and (36) that the kernel of operator \(H_{\omega , v}\) includes the subspace Z.

Based on the above analysis, we give the following assumption.

Assumption 3.1

Spectral decomposition of operator \(H_{\omega , v}\): in terms of the space X, it can make the following direct sum decomposition

$$\begin{aligned} X=N+Z+P, \end{aligned}$$
(37)

where Z has been defined by \(Z=\left\{ k_{1} T_{1}^{\prime }(0) {\hat{a}}(x)+\right. \left. k_{2} T_{2}^{\prime }(0) {\hat{a}}(x) \mid k_{1}, k_{2} \in {\mathbb {R}}\right\} \), and N is a subspace of X with finite dimensions, which satisfies

$$\begin{aligned} \left\langle H_{\omega , v} q, q\right\rangle <0, \quad \forall 0 \ne q \in N, \end{aligned}$$
(38)

P is a closed subspace of X, for arbitrary \(q \in P\), there is a constant \(\tau > 0\) independent of q satisfying

$$\begin{aligned} \left\langle H_{\omega , v} q, q\right\rangle \geqslant \tau \Vert q\Vert _{X}^{2}, \quad \forall q \in P. \end{aligned}$$
(39)

We define the map** \(d(\omega , v): {\mathbb {R}} \times {\mathbb {R}} \rightarrow {\mathbb {R}}\) in the form of

$$\begin{aligned} d(\omega , v)=E\left( {\hat{a}}_{\omega , v}\right) -\omega Q_{1}\left( {\hat{a}}_{\omega , v}\right) -v Q_{2}\left( {\hat{a}}_{\omega , v}\right) , \end{aligned}$$
(40)

and the corresponding Hessian matrix is expressed as \(d^{\prime \prime }(\omega , v)\), which possesses the bilinear type. Furthermore, \(p (d^{\prime \prime })\) is used to represent the positive-eigenvalue’s number of \(d^{\prime \prime }\), and \(n (H_{\omega , v})\) is used to represent the negative-eigenvalue’s number of \(H_{\omega , v}\).

According to [23], we know that the local solution of initial value problem (15)–(16) of EK equation exists. Based on the above discussion, it can be seen that equation (15) has three conserved quantities satisfying (26) and (27), and the solitary wave satisfies (32). Moreover, we define the operator \(H_{\omega , v}\). Therefore, based on the "stability theorem" in the introduction part of [22] or Theorem 4.1 in [22], for the solitary wave of equation (15), we can get the following lemmas.

Lemma 3.1

(Stability Theorem) Given \(\omega \in {\mathfrak {g}}\), consider the scalar function d restricted to the centralizer \({\mathfrak {g}}_{\omega }\). Assume that it is non-degenerate at \(\omega \) and let \(p\left( d^{\prime \prime }\right) \) be the number of positive eigenvalues of its Hessian at \(\omega \). Let n(H) be the number ofnegative eigenvalues of \(H_{\omega }\). Then \(p\left( d^{\prime \prime }\right) \le n(H)\). If \(p\left( d^{\prime \prime }\right) = n(H)\), then the solitary wave is \(G_\omega \)-stable, where \(G_\omega \) is the centralizer subgroup.

According to the above lemma, we can obtain the following abstract orbital stability theorem for solitary wave of equation (15).

Theorem 3.2

Assume that there exist three functionals \(E(q), Q_{1}(q), Q_{2}(q)\) satisfying (26) and (27), and the solitary wave \(T_{1}(\omega t) T_{2}(v t) {\hat{a}}_{\omega , v}(x)\) satisfying (32). Moreover, the operator \(H_{\omega , v}\) defined by (33) satisfies Assumption 3.1. Therefore, if \(d(\omega , v)\) is nonsingular, \(p (d^{\prime \prime })=n (H_{\omega , v})\), thus the solitary wave \(T_{1}(\omega t) T_{2}(v t) {\hat{a}}_{\omega , v}(x)\) of EK equation (15) is orbitally stable.

According to Theorem 3.2, in terms of the solitary wave of EK equation (15), we can further derive the following result on its orbital stability.

Theorem 3.3

If \(\sigma <0\) and \(\omega , v\) satisfy \(4\omega + v^{2}<0\), the solitary wave \(e^{-i \omega t} {\hat{a}}(x-v t)\) of equation (15) is orbitally stable.

4 Proof of Theorem 3.3

According to Theorem 3.2 and (23)–(33), we only need to prove that operator \(H_{\omega , v}\) defined by (33) satisfies Assumption 3.1 and \(p (d^{\prime \prime })=n (H_{\omega , v})\), then it will be obtained from Theorem 3.2 that the solitary wave of EK equation (15) is orbitally stable.

4.1 Proof of Assumption 3.1

Firstly, the spectral decomposition of \(H_{\omega , v}\) is studied, including the spectral properties of per decomposition component.

For arbitrary \(\phi (x) \in X\), assume that

$$\begin{aligned} \phi (x)=e^{i \psi (x)} r(x), \quad r(x)=r_{1}(x)+i r_{2}(x), \quad r_{1}(x)={\text {Re}} r(x), \end{aligned}$$
(41)

thus we have

$$\begin{aligned} H_{\omega , v} \phi =\left[ L_{11} r_{1}+L_{12} r_{2}+i\left( L_{21} r_{1}+L_{22} r_{2}\right) \right] e^{i \psi }, \end{aligned}$$

where

$$\begin{aligned} L_{11}=-\frac{\partial ^{2}}{\partial x^{2}}+\left( \psi ^{\prime }\right) ^{2}-g(a^{2})-2 g^{\prime }(a^{2}) a^{2}-\omega -v \psi ^{\prime }+20 \delta ^{2} a^{4}+12 \delta a^{2} \psi ^{\prime }-6 \delta v a^{2},\\ L_{12}=\psi ^{\prime \prime }+2 \delta a^{2} \frac{\partial }{\partial x},\\ L_{21}=-\psi ^{\prime \prime }-2 \delta a^{2} \frac{\partial }{\partial x}-8 \delta a a^{\prime },\\ L_{22}=-\frac{\partial ^{2}}{\partial x^{2}}+\left( \psi ^{\prime }\right) ^{2}-g(a^{2})-\omega -v \psi ^{\prime }. \end{aligned}$$

Since

$$\begin{aligned} \left\langle H_{\omega , v} \phi , \phi \right\rangle =\left\langle L_{11} r_{1}, r_{1}\right\rangle +\left\langle L_{12} r_{2}, r_{1}\right\rangle +\left\langle L_{21} r_{1}, r_{2}\right\rangle +\left\langle L_{22} r_{2}, r_{2}\right\rangle , \end{aligned}$$

where

$$\begin{aligned} \begin{array}{l} \left\langle L_{11} r_{1}, r_{1}\right\rangle +\left\langle L_{12} r_{2}, r_{1}\right\rangle +\left\langle L_{21} r_{1}, r_{2}\right\rangle \\ =\left\langle L_{11} r_{1}, r_{1}\right\rangle +\left\langle 2 \delta a^{2} r_{2}^{\prime }, r_{1}\right\rangle +\left\langle -2 \delta a^{2} r_{1}^{\prime }, r_{2}\right\rangle +\left\langle -8 \delta a a^{\prime } r_{1}, r_{2}\right\rangle , \end{array} \end{aligned}$$

due to

$$\begin{aligned} \begin{aligned} \left\langle -2 \delta a^{2} r_{1}^{\prime }, r_{2}\right\rangle&=-2 \delta \int \limits _{{\mathbb {R}}} a^{2} r_{2} d r_{1} \\&=2 \delta \int \limits _{{\mathbb {R}}} r_{1}\left( 2 a a^{\prime } r_{2}+a^{2} r_{2}^{\prime }\right) d x \\&=\left\langle 4 \delta a a^{\prime } r_{2}, r_{1}\right\rangle +\left\langle 2 \delta a^{2} r_{2}^{\prime }, r_{1}\right\rangle , \end{aligned}\\ 20 \delta ^{2} a^{4}+12 \delta a^{2} \psi ^{\prime }-6 \delta v a^{2}=8 \delta ^{2} a^{4}\left( \psi ^{\prime }=v / 2-\delta a^{2}\right) , \end{aligned}$$

we have

$$\begin{aligned} \left\langle L_{12} r_{2}, r_{1}\right\rangle +\left\langle L_{21} r_{1}, r_{2}\right\rangle =\left\langle 4 \delta a^{2} r_{2}^{\prime }, r_{1}\right\rangle +\left\langle -4 \delta a a^{\prime } r_{1}, r_{2}\right\rangle ,\\ L_{11}={\bar{L}}_{11}+4 \delta ^{2} a^{4}, \end{aligned}$$

where

$$\begin{aligned} {\bar{L}}_{11}=-\frac{\partial ^{2}}{\partial x^{2}}+\left( \psi ^{\prime }\right) ^{2}-g(a^{2})-2 g^{\prime }(a^{2}) a^{2}-\omega -v \psi ^{\prime }+4 \delta ^{2} a^{4}. \end{aligned}$$
(42)

Substitute the above into \(\left\langle H_{\omega , v} \phi , \phi \right\rangle \), we can obtain

$$\begin{aligned} \left\langle H_{\omega , v} \phi , \phi \right\rangle =\left\langle {\bar{L}}_{11} r_{1}, r_{1}\right\rangle +\left\langle L_{22} r_{2}, r_{2}\right\rangle +\left\langle 4 \delta a^{2} r_{2}^{\prime }, r_{1}\right\rangle +\left\langle 4 \delta ^{2} a^{4} r_{1}-4 \delta a a^{\prime } r_{2}, r_{1}\right\rangle , \end{aligned}$$
(43)

From (6) we get

$$\begin{aligned} L_{22} a(x)=0, \end{aligned}$$
(44)

then we differentiate (6) with respect to x and combine with (9) and (42), it can be deduced that

$$\begin{aligned} {\bar{L}}_{11} a^{\prime }(x)=0. \end{aligned}$$
(45)

According to (14), it can be seen that \(x=0\) is the unique simple zero of \({\hat{a}}(x)\) and \({\hat{a}}(x)\) varies the sign for only one time. Then from the Sturm–Liouville theorem, we know that zero is the second eigenvalue of \({\bar{L}}_{11}\), and there exists only one negative eigenvalue \(-\lambda _{11}^{2}\) for \({\bar{L}}_{11}\), the corresponding eigenfunction is \(\chi _{11}\), which means

$$\begin{aligned} {\bar{L}}_{11} \chi _{11}=-\lambda _{11}^{2} \chi _{11}. \end{aligned}$$
(46)

In addition, since \(l=-\omega -\frac{v^{2}}{4}\), \({\bar{L}}_{11}\) can be expressed as

$$\begin{aligned} {\bar{L}}_{11}=-\frac{\partial ^{2}}{\partial x^{2}}+l+M_{1}(x), \end{aligned}$$
(47)

where

$$\begin{aligned} M_{1}(x)=5 \delta ^{2} a^{4}-g(a^{2})-2 g^{\prime }(a^{2}) a^{2}. \end{aligned}$$

From (14), we can see that when \(|x |\rightarrow \infty \), \(a^{2} \rightarrow 0\). Therefore, we obtain

$$\begin{aligned} M_{1}(x) \rightarrow 0 \quad (|x |\rightarrow \infty ). \end{aligned}$$
(48)

On the basis of (47)–(48) and the Weyl’s essential spectral theorem of [24], it can be given that

$$\begin{aligned} \sigma _{\textrm{ess}}\left( {\bar{L}}_{11}\right) =[l,+\infty ), \quad l>0. \end{aligned}$$
(49)

Then we can obtain the following spectral properties of \({\bar{L}}_{11}\).

Proposition 4.1

For \({\bar{L}}_{11}\), its kernel is spanned by \(a^{\prime }(x)\), and it has unique negative simple eigenvalue, the remaining spectrum is positive, which is bounded away from zero.

According to [21] and the above Proposition 4.1, for arbitrary real-valued function \(r_{1} \in H^{1}({\mathbb {R}})\), when the following equality holds,

$$\begin{aligned} \left\langle r_{1}, a^{\prime }\right\rangle =\left\langle r_{1}, \chi _{11}\right\rangle =0, \end{aligned}$$
(50)

there is a positive real number \({\bar{\tau }}_{1}>0\) independent of \(r_{1}\), which satisfies

$$\begin{aligned} \left\langle {\bar{L}}_{11} r_{1}, r_{1}\right\rangle \geqslant {\bar{\tau }}_{1}\left\| r_{1}\right\| _{L^{2}}^{2}. \end{aligned}$$
(51)

Further, from (47)–(51), we can get the following lemma (see Appendix in [25]).

Lemma 4.1

For arbitrary real-valued function \(r_{1} \in H^{1}({\mathbb {R}})\), when (50) holds, there is a positive constant \(\tau _{1}>0\) independent of \(r_{1}\), which satisfies

$$\begin{aligned} \left\langle {\bar{L}}_{11} r_{1}, r_{1}\right\rangle \geqslant \tau _{1}\left\| r_{1}\right\| _{H^{1}}^{2}. \end{aligned}$$
(52)

Next, \(L_{22}\) will be discussed.

It can be seen from (14) and (44) that a(x) does not change sign, the first eigenvalue of \(L_{22}\) is 0, and \(L_{22}\) can also be expressed as

$$\begin{aligned} L_{22}=-\frac{\partial ^{2}}{\partial x^{2}}+l+M_{2}(x), \end{aligned}$$
(53)

where \(M_{2}(x)=\delta ^{2} a^{4}-g(a^{2})\).

Since \(a^{2} \rightarrow 0\) when \(|x |\rightarrow \infty \), we get

$$\begin{aligned} M_{2}(x) \rightarrow 0 \quad (|x |\rightarrow \infty ). \end{aligned}$$
(54)

Further, we have

$$\begin{aligned} \sigma _{\textrm{ess}}\left( L_{22}\right) =[l,+\infty ), \quad l>0. \end{aligned}$$
(55)

Therefore, we can obtain the spectral properties of \(L_{22}\) as follows.

Proposition 4.2

For \(L_{22}\), its kernel is spanned by a(x), and the remaining spectrum of \(L_{22}\) is positive, which is bounded away from zero.

Similarly, according to Proposition 4.2 and (53)–(55), we can get the following lemma.

Lemma 4.2

For arbitrary real-valued function \(r_{2} \in H^{1}({\mathbb {R}})\), when the following equality holds,

$$\begin{aligned} \left\langle r_{2}, a\right\rangle =0, \end{aligned}$$
(56)

there is a positive number \(\tau _{2}>0\) independent of \(r_{2}\), which satisfies

$$\begin{aligned} \left\langle L_{22} r_{2}, r_{2}\right\rangle \geqslant \tau _{2}\left\| r_{2}\right\| _{H^{1}}^{2}. \end{aligned}$$
(57)

In what follows, we will prove that \(n (H_{\omega , v})= 1\) and Assumption 3.1 holds.

For arbitrary \(\phi (x) \in X\), assume that

$$\begin{aligned} \phi (x)=e^{i \psi (x)}\left( r_{1}(x)+i r_{2}(x)\right) , \quad r_{2}(x)=a(x) r_{3}(x), \end{aligned}$$
(58)

where \(r_{1}, r_{2}, r_{3}\) are real-valued functions, \(r_{1}, r_{2} \in H^{1}({\mathbb {R}})\). Note that

$$\begin{aligned} \begin{aligned} \left\langle L_{22} r_{2}(x), r_{2}(x)\right\rangle&=\left\langle -r_{2}^{\prime \prime }, r_{2}\right\rangle +\left\langle d r_{2}, r_{2}\right\rangle +\left\langle M_{2} r_{2}, r_{2}\right\rangle \\&=\left\langle -a^{\prime \prime } r_{3}+d a r_{3}+M_{2} a r_{3}, a r_{3}\right\rangle +\left\langle -2 a^{\prime } r_{3}^{\prime }-a r_{3}^{\prime \prime }, a r_{3}\right\rangle \\&=\left\langle L_{22} a, a r_{3}^{2}\right\rangle -\left\langle \left( a^{2} r_{3}^{\prime }\right) ^{\prime }, r_{3}\right\rangle \\&=-\int \limits _{{\mathbb {R}}} r_{3} d\left( a^{2} r_{3}^{\prime }\right) \\&=\left\langle a r_{3}^{\prime }, a r_{3}^{\prime }\right\rangle , \end{aligned} \end{aligned}$$
(59)

then according to (43), (58) and (59), we can obtain

$$\begin{aligned} \begin{aligned} \left\langle H_{\omega , v} \phi , \phi \right\rangle&=\left\langle {\bar{L}}_{11} r_{1}, r_{1}\right\rangle +\left\langle a r_{3}^{\prime }, a r_{3}^{\prime }\right\rangle +\left\langle 4 \delta ^{2} a^{4} r_{1}-4 \delta a a^{\prime } r_{2}, r_{1}\right\rangle +\left\langle 4 \delta a^{2} r_{2}^{\prime }, r_{1}\right\rangle \\&=\left\langle {\bar{L}}_{11} r_{1}, r_{1}\right\rangle +\int \limits _{{\mathbb {R}}}\left( \left( a r_{3}^{\prime }\right) ^{2}+2\left( a r_{3}^{\prime }\right) \left( 2 \delta a^{2} r_{1}\right) +\left( 2 \delta a^{2} r_{1}\right) ^{2}\right) d x \\&=\left\langle {\bar{L}}_{11} r_{1}, r_{1}\right\rangle +\int \limits _{{\mathbb {R}}}\left( a r_{3}^{\prime }+2 \delta a^{2} r_{1}\right) ^{2} d x. \end{aligned} \end{aligned}$$
(60)

Take

$$\begin{aligned}{} & {} \chi _{-}=\left( \chi _{11}+i \chi _{12}\right) e^{i \psi (x)}, \end{aligned}$$
(61)
$$\begin{aligned}{} & {} \chi _{12}=a \chi _{13}=a\left( -2 \delta \int \limits _{-\infty }^{x} a(s) \chi _{11}(s) d s+k_{1}\right) , \end{aligned}$$
(62)

where \(\forall k_{1} \in {\mathbb {R}}\), due to (58) and (60)–(62), we can get

$$\begin{aligned} \left\langle H_{\omega , v} \chi _{-}, \chi _{-}\right\rangle =\left\langle {\bar{L}}_{11} \chi _{11}, \chi _{11}\right\rangle =-\lambda _{11}^{2}<0. \end{aligned}$$
(63)

Choose \(k_{1}\) to make

$$\begin{aligned} \left\langle \chi _{12}, a\right\rangle =0, \end{aligned}$$
(64)

and let

$$\begin{aligned} N=\left\{ k \chi _{-} \mid k \in {\mathbb {R}}\right\} , \end{aligned}$$
(65)

therefore, according to (63) and (65), we can prove that (38) holds.

Take

$$\begin{aligned}{} & {} \chi _{1}=\left[ a^{\prime }(x)+i a(x)\left( k_{2}-\delta a^{2}(x)\right) \right] e^{i \psi (x)}, \end{aligned}$$
(66)
$$\begin{aligned}{} & {} \chi _{2}=i a(x) e^{i \psi (x)}, \end{aligned}$$
(67)

and choose \(k_{2}\) to make

$$\begin{aligned} \left\langle \left( k_{2}-\delta a^{2}\right) a, a\right\rangle =0, \end{aligned}$$
(68)

thus we can write the subspace Z as

$$\begin{aligned} Z=\left\{ k_{3} \chi _{1}+k_{4} \chi _{2} \mid k_{3}, k_{4} \in {\mathbb {R}}\right\} . \end{aligned}$$
(69)

Moreover, define the subspace P as

$$\begin{aligned} P=\left\{ p \in X \mid p=\left( p_{1}+i p_{2}\right) e^{i \psi (x)},\left\langle p_{1}, \chi _{11}\right\rangle =\left\langle p_{1}, a^{\prime }\right\rangle =\left\langle p_{2}, a\right\rangle =0\right\} , \end{aligned}$$
(70)

then we can obtain the lemma as follows:

Lemma 4.3

For arbitrary \(p \in P\), there is a positive number \(\tau >0\) independent of p, which satisfies

$$\begin{aligned} \left\langle H_{\omega , v} p, p\right\rangle \geqslant \tau \Vert p\Vert _{H^{1}}^{2}. \end{aligned}$$
(71)

Proof

For \(\forall p=\left( p_{1}+i p_{2}\right) e^{i \psi (x)}\), let \(p_{2}=a p_{3}\), where \(p_{1}, p_{2}, p_{3}\) are real functions and \(p_{1}, p_{2} \in H^{1}({\mathbb {R}})\). From (60), we have

$$\begin{aligned} \left\langle H_{\omega , v} p, p\right\rangle =\left\langle {\bar{L}}_{11} p_{1}, p_{1}\right\rangle +\int \limits _{{\mathbb {R}}}\left( a p_{3}^{\prime }+2 \delta a^{2} p_{1}\right) ^{2} d x. \end{aligned}$$
(72)

According to (70), Lemma 4.1 and Lemma 4.2, we can obtain

$$\begin{aligned} \left\langle {\bar{L}}_{11} p_{1}, p_{1}\right\rangle \geqslant \tau _{1}\left\| p_{1}\right\| _{H^{1}}^{2}, \quad \left\langle L_{22} p_{2}, p_{2}\right\rangle \geqslant \tau _{2}\left\| p_{2}\right\| _{H^{1}}^{2}. \end{aligned}$$
(73)
  1. (1)

    If \(\left\| a p_{3}^{\prime }\right\| _{L_{2}} \geqslant C_{0}\left\| p_{1}\right\| _{L_{2}}, C_{0}=4 \delta ^{2}|a(0) |^{2}\), we get

    $$\begin{aligned} \begin{aligned} \int \limits _{{\mathbb {R}}}\left( a p_{3}^{\prime }+2 \delta a^{2} p_{1}\right) ^{2} d x&\geqslant \frac{\left\| a p_{3}^{\prime }\right\| _{L_{2}}^{2}}{2}-\int \limits _{{\mathbb {R}}}\left( 2 \delta a^{2} p_{1}\right) ^{2} d x \\&\geqslant \frac{\left\| a p_{3}^{\prime }\right\| _{L_{2}}^{2}}{4}=\frac{\left\langle L_{22} p_{2}, p_{2}\right\rangle }{4}, \end{aligned} \end{aligned}$$

    then from (72) and (73), we can obtain

    $$\begin{aligned} \left\langle H_{\omega , v} p, p\right\rangle \geqslant \tau _{1}\left\| p_{1}\right\| _{H^{1}}^{2}+\frac{\tau _{2}}{4}\left\| p_{2}\right\| _{H^{1}}^{2} \geqslant \tau \Vert p\Vert _{H^{1}}^{2}. \end{aligned}$$
  2. (2)

    If \(\left\| a p_{3}^{\prime }\right\| _{L_{2}} \le C_{0}\left\| p_{1}\right\| _{L_{2}}, C_{0}=4 \delta ^{2}|a(0) |^{2}\), according to (72), (73) and (59), we can obtain

    $$\begin{aligned} \begin{aligned} \left\langle H_{\omega , v} p, p\right\rangle&\geqslant \tau _{1}\left\| p_{1}\right\| _{H^{1}}^{2} \geqslant \frac{\tau _{1}}{2}\left\| p_{1}\right\| _{H^{1}}^{2}+\frac{\tau _{1}}{2 C_{0}^{2}}\left\| a p_{3}^{\prime }\right\| _{L_{2}}^{2} \\&\geqslant \frac{\tau _{1}}{2}\left\| p_{1}\right\| _{H^{1}}^{2}+\frac{\tau _{1} \tau _{2}}{2 C_{0}^{2}}\left\| p_{2}\right\| _{H^{1}}^{2} \geqslant \tau \Vert p\Vert _{H^{1}}^{2}. \end{aligned} \end{aligned}$$

Thus, Lemma 4.3 has been proved. \(\square \)

For arbitrary \(\phi (x)=e^{i \psi (x)}\left( r_{1}+i r_{2}\right) \in X\), we take

$$\begin{aligned} \alpha =\left\langle r_{1}, \chi _{11}\right\rangle , \quad \beta _{1}=\frac{\left\langle r_{1}, a^{\prime }\right\rangle }{\left\| a^{\prime }\right\| _{L^{2}}^{2}}, \quad \beta _{2}=\frac{\left\langle r_{2}, a\right\rangle }{\Vert a\Vert _{L^{2}}^{2}}, \end{aligned}$$
(74)

thus we can express \(\phi (x)\) as

$$\begin{aligned} \phi (x)=\alpha \chi _{-}+\beta _{1} \chi _{1}+\beta _{2} \chi _{2}+p. \end{aligned}$$
(75)

According to (45), (46) and (64)–(68), we know that \(\phi (x)\) can be uniquely represented by (73). Therefore, from (65) and (69)–(75), we can obtain that \(n\left( H_{\omega , v}\right) =1\) and Assumption 3.1 holds.

4.2 Proof of \(p\left( d^{\prime \prime }\right) =n\left( H_{\omega , v}\right) \)

We have proved that \(n\left( H_{\omega , v}\right) =1\) in Sect. 4.1, and in this section, under the conditions of Theorem 3.3, we just need to prove \(p\left( d^{\prime \prime }\right) =1\), then it can be obtained that \(p\left( d^{\prime \prime }\right) =n\left( H_{\omega , v}\right) =1\).

Since

$$\begin{aligned} d(\omega , v)=E\left( {\hat{a}}\right) -\omega Q_{1}\left( {\hat{a}}\right) -v Q_{2}\left( {\hat{a}}\right) , \end{aligned}$$

we can get

$$\begin{aligned} d_{\omega }=-Q_{1}({\hat{a}}), \quad d_{v}=-Q_{2}({\hat{a}}),\\ d_{\omega v}=d_{v \omega }=-\left\langle Q_{1}^{\prime }({\hat{a}}), \frac{\partial {\hat{a}}}{\partial v}\right\rangle =-\left\langle {\hat{a}}, \frac{\partial {\hat{a}}}{\partial v}\right\rangle =-\frac{1}{2} \frac{\partial }{\partial v}\langle a, a\rangle ,\\ d_{\omega \omega }=-\left\langle Q_{1}^{\prime }({\hat{a}}), \frac{\partial {\hat{a}}}{\partial \omega }\right\rangle =-\left\langle {\hat{a}}, \frac{\partial {\hat{a}}}{\partial \omega }\right\rangle =-\frac{1}{2} \frac{\partial }{\partial \omega }\langle a, a\rangle ,\\ \begin{aligned} d_{v v}&=-\left\langle Q_{2}^{\prime }({\hat{a}}), \frac{\partial {\hat{a}}}{\partial v}\right\rangle =-\left\langle -i {\hat{a}}_{x}+2 \delta a^{2} {\hat{a}}, \frac{\partial {\hat{a}}}{\partial v}\right\rangle \\&=-\left\langle a \psi ^{\prime }-i a^{\prime }+2 \delta a^{3},-i a \frac{\partial \psi }{\partial v}+\frac{\partial a}{\partial v}\right\rangle \\&=-\left\langle a \psi ^{\prime }, \frac{\partial a}{\partial v}\right\rangle +\left\langle a^{\prime }, a \frac{\partial \psi }{\partial v}\right\rangle -\left\langle 2 \delta a^{3}, \frac{\partial a}{\partial v}\right\rangle \\&=-\left\langle \frac{v}{2} a, \frac{\partial a}{\partial v}\right\rangle -\left\langle \delta a^{3}, \frac{\partial a}{\partial v}\right\rangle +\left\langle a^{\prime }, a \frac{\partial \psi }{\partial v}\right\rangle \\&=-\frac{v}{2} \int \limits _{{\mathbb {R}}} a \frac{\partial a}{\partial v} d x-\delta \int \limits _{{\mathbb {R}}} a^{3} \frac{\partial a}{\partial v} d x+\frac{1}{2} \int \limits _{{\mathbb {R}}} \frac{\partial \psi }{\partial v} d a^{2} \\&=-\frac{v}{2} \int \limits _{{\mathbb {R}}} a \frac{\partial a}{\partial v} d x-\delta \int \limits _{{\mathbb {R}}} a^{3} \frac{\partial a}{\partial v} d x-\frac{1}{2} \int \limits _{{\mathbb {R}}} a^{2}\left( \frac{1}{2}-2 \delta a \frac{\partial a}{\partial v}\right) d x \\&=-\frac{v}{4} \frac{\partial }{\partial v}\langle a, a\rangle -\frac{1}{4}\langle a, a\rangle . \end{aligned} \end{aligned}$$

Therefore, we can obtain

$$\begin{aligned} d^{\prime \prime }=\left( \begin{array}{ll} d_{\omega \omega } &{} d_{\omega v} \\ d_{v \omega } &{} d_{v v} \end{array}\right) =\left( \begin{array}{cc} -\frac{1}{2} \frac{\partial }{\partial \omega }\langle a, a\rangle &{} -\frac{1}{2} \frac{\partial }{\partial v}\langle a, a\rangle \\ -\frac{1}{2} \frac{\partial }{\partial v}\langle a, a\rangle &{} -\frac{v}{4} \frac{\partial }{\partial v}\langle a, a\rangle -\frac{1}{4}\langle a, a\rangle \end{array}\right) . \end{aligned}$$

Further, we can compute

$$\begin{aligned} {\text {det}}\left( d^{\prime \prime }\right) =\frac{v}{8} \frac{\partial }{\partial \omega }\langle a, a\rangle \frac{\partial }{\partial v}\langle a, a\rangle +\frac{1}{8}\langle a, a\rangle \frac{\partial }{\partial \omega }\langle a, a\rangle -\frac{1}{4}\left( \frac{\partial }{\partial v}\langle a, a\rangle \right) ^{2}. \end{aligned}$$
(76)

Next we prove \(p\left( d^{\prime \prime }\right) =1\), i.e. \({\text {det}}\left( d^{\prime \prime }\right) <0\).

According to the conditions \(4 \omega +v^{2}<0\) and \(\sigma <0\) in Theorem 3.3, we have \(\sigma <0, l=-\frac{\left( 4 \omega +v^{2}\right) }{4}>0\). It can be seen from (14) that

$$\begin{aligned} a^{2}(x)=-\frac{l}{\sigma } {\text {sech}}^{2}(\sqrt{l} x). \end{aligned}$$

Then we have

$$\begin{aligned} \langle a, a\rangle =-\frac{l}{\sigma } \int \limits _{{\mathbb {R}}} {\text {sech}}^{2}(\sqrt{l} x) d x=-\frac{l}{\sigma } \cdot \frac{2}{\sqrt{l}}=-\frac{2 \sqrt{l}}{\sigma }>0,\\ \frac{\partial }{\partial \omega }\langle a, a\rangle =-\frac{2}{\sigma } \cdot \frac{1}{2 \sqrt{l}} \cdot \frac{\partial l}{\partial \omega }=\frac{1}{\sigma \sqrt{l}}<0,\\ \frac{\partial }{\partial v}\langle a, a\rangle =-\frac{2}{\sigma } \cdot \frac{1}{2 \sqrt{l}} \cdot \frac{\partial l}{\partial v}=\frac{v}{2 \sigma \sqrt{l}}, \end{aligned}$$

According to (76), we can get

$$\begin{aligned} \begin{aligned} {\text {det}}\left( d^{\prime \prime }\right)&=\frac{v}{8} \frac{\partial }{\partial \omega }\langle a, a\rangle \frac{\partial }{\partial v}\langle a, a\rangle +\frac{1}{8}\langle a, a\rangle \frac{\partial }{\partial \omega }\langle a, a\rangle -\frac{1}{4}\left( \frac{\partial }{\partial v}\langle a, a\rangle \right) ^{2} \\&=-\frac{1}{4 \sigma ^{2}}<0. \end{aligned} \end{aligned}$$

So we get \({\text {det}}\left( d^{\prime \prime }\right) <0\) under the conditions \(4 \omega +v^{2}<0\) and \(\sigma <0\). Further, we obtain \(p\left( d^{\prime \prime }\right) =1\), i.e. \(p\left( d^{\prime \prime }\right) =n (H_{\omega , v})\).

To sum up, under the conditions \(4 \omega +v^{2}<0\) and \(\sigma <0\) in Theorem 3.3, we prove that \(p\left( d^{\prime \prime }\right) =n (H_{\omega , v})\). Therefore, Theorem 3.3 is proved completely, that is, the solitary wave \(e^{-i \omega t} {\hat{a}}(x-v t)\) of equation (15) is orbitally stable.