The Dirichlet problem with sublinear indefinite nonlinearities

Nonlinear Analysis 73 (2010) 2831–2841 Contents lists available at ScienceDirect Nonlinear Analysis journal homepage: www.elsevier.com/locate/na The Dirichlet problem with sublinear indefinite nonlinearities Shumao Liu ∗ School of Mathematic Sciences, Peking University, Beijing, 100871, PR China School of Applied Mathematics, Central University of Finance and Economics, Beijing, 100081, PR China a r t i c l e i n f o Article history: Received 1 February 2010 Accepted 10 June 2010 Keywords: Sublinear elliptic equations Indefinite nonlinearities Nodal solution Morse inequality a b s t r a c t We consider the Dirichlet problem with sublinear indefinite nonlinearities −∆u − λu = h−(x)g1(u)+h+(x)g2(u)with h−(x) ≤ 0 and h+(x) ≥ 0. Using decomposition of the space H10 (Ω), we construct a special deformation, and then compute the Morse critical groups at zero of the energy functional. Combining with the Morse critical groups at infinity, some existences, such as the existence of a nodal solution, are established via Morse theory. © 2010 Elsevier Ltd. All rights reserved. 1. Introduction In this paper, we are concerned with a class of nonlinear elliptic Dirichlet problems involving sublinear indefinite nonlinearities:{−∆u− λu = g(x, u) inΩ, u = 0 on ∂Ω, (1.1) whereΩ ⊂ Rn is a bounded domain with smooth boundary, g(x, u) = h−(x)g1(u)+ h+(x)g2(u), h−(x) ≤ 0, h+(x) ≥ 0, and gi(x, u) (i = 1, 2) satisfies the condition gi(x, u) = o(|u|) as u→ 0, for i = 1, 2. For a continuous function h, set Ω0 = {x ∈ Ω|h(x) = 0}, Ω− = {x ∈ Ω|h(x) < 0}, Ω+ = {x ∈ Ω|h(x) > 0}. Let σ(Ω0) and σ(Ω) be the set of eigenvalues of −∆ on Ω0 and Ω with Dirichlet boundary value, and λ1(Ω0), λ1(Ω) be the first eigenvalue in σ(Ω0) and σ(Ω), respectively. λ1(Ω0) and the number λ∗(h) := inf { ‖∇u‖2 ∣∣∣∣u ∈ H10 (Ω), ‖u‖2 = 1, ∫ Ω h(x)|u|qdx = 0 } play important roles in the study of the indefinite nonlinear problems. Obviously, λ1(Ω) ≤ λ∗(h) ≤ λ1(Ω0) (see [1, p. 1443]). There exists h(x) such that λ∗(h) can be arbitrarily large. An example is given in [2, p. 112]. Throughout the paper, we assume that h has a ‘‘thick’’ zero set, i.e., Ω− ∩Ω+ = ∅. (H) Under this condition, λ∗(h) < λ1(Ω0) (see [2, p. 98]). ∗ Corresponding address: School of Mathematic Sciences, Peking University, Beijing, 100871, PR China. E-mail address: Lioushumao@163.com. 0362-546X/$ – see front matter© 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.na.2010.06.024 2832 S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 Sublinear indefinite problems of this type arise in population dynamics (see, e.g., [3]). They describe the stationary behavior of a population in a heterogeneous environment (see [3,4,1]). A number of papers have appeared that deal with Dirichlet problems with indefinite nonlinearity (see, e.g., [2,5,6,4,1,7]). For superlinear indefinite nonlinearity, there is extensive literature on nontrivial solutions of problemof the type (1.1) (see, e.g., [2,5,6] and references therein). For sublinear problems, most papers deal with the existence of positive solutions of problem (1.1). The existence and bifurcation of nontrivial nonnegative solutions for varying λ have been studied by Alama [4]. Using direct homotopic constructions and saddle point arguments, Moroz [1] considered the following sublinear indefinite Dirichlet problem:{−∆u− λu = h(x)u|u|q−2 + f (x, u) inΩ, u = 0 on ∂Ω, where q ∈ (1, 2), f (x, u) = o(|u|), as |u| → 0. He computed the Morse critical groups at zero for the corresponding functional when λ < λ∗(h) and |f (x, u)| ≤ C(1 + |u|p−1) with 2 < p < 2∗ and found that all critical groups at zero are trivial, where 2∗ = 2NN+2 if N ≥ 3 and 2∗ = ∞ if N = 2. Then he obtained a nontrivial (possibly sign-changing) solution of (1.1) when the nonlinearity f (x, u) is asymptotically linear. We are interested in the existence and nodal properties of the nontrivial weak solutions of (1.1) when both int(Ω+) and int(Ω−) are nonempty sets. The approach of this paper is inspired by thework of Chang and Jiang [6] on superlinear problems and that of Moroz [1] on sublinear problems. Set h−(x) = min{0, h(x)}, h+(x) = max{0, h(x)}. Using the decomposition lemma of H10 (Ω) introduced in [6], we can split the nonlinear term of the corresponding functional into ∫ Ω h−(x)G1(v)dx and ∫ Ω h+(x)G2(w)dx with G1 = ∫ v 0 g1(s)ds and G2(w) = ∫ w 0 g2(s)ds. Recomposing the proof in [1], we can compute the critical groups at zero under weaker conditions on λ. In fact, we can replace the condition λ < λ∗(h) by λ < λ1(Ω0) under condition (H). Our main result reads as follows. Theorem 1.1. Let g1, g2 : Ω × R→ R be Carathéodory functions satisfying (g1) lim|t|→∞ gi(t)t = 0, uniformly for x ∈ Ω, i = 1, 2; (g2) lim|t|→0 gi(t)tq−1 = C1 > 0, uniformly for x ∈ Ω, i = 1, 2, where q ∈ (1, 2). Let h be a continuous function satisfying the ‘‘thick’’ zero set condition (H). Furthermore, we assume the number of positive and negative solutions of (1.1) is finite. If λ < λ1(Ω0) and λ 6∈ σ(Ω), Eq. (1.1) has at least three nontrivial solutions. Among them, one is positive, one is negative, and one is a sign-changing solution. Remark 1.2. Noting the results of [8] about the symmetric sublinear elliptic Dirichlet problem, the assumption that the positive and negative solutions are finite in number is not superfluous. Remark 1.3. In comparing with the results in [4,1], the following two aspects are new. First, the assumption λ < λ∗(h) is improved by λ < λ1(Ω0). Second, we find a nodal solution. The rest of the paper is organized as follows: in thenext section,we first prove somekindof (P.S.) condition (for definition, see Proposition 2.4 below) and some preliminary propositions. In Section 3, we compute the critical groups at zero and infinity, and give the proof of Theorem 1.1. 2. Some preliminary propositions In this section, using the decomposition lemma of [6], we shall prove some kind of (P.S.) condition and some preliminary propositions. Throughout this paper, we set ‖ · ‖2 as the usual L2(Ω) norm and ‖ · ‖ as the usual H10 (Ω) norm; i.e., ‖u‖ = ‖∇u‖2. Consider the corresponding functional associated with Eq. (1.1): I(u) = 1 2 ∫ Ω (|∇u|2 − λu2)dx− ∫ Ω h−(x)G1(u)dx− ∫ Ω h+(x)G2(u)dx, where Gi(x, u) = ∫ u 0 gi(x, s)ds, i = 1, 2. In order to split the nonlinear terms, we note thatH10 (Ω−),H10 (Ω0) andH10 (Ω+) are orthogonal each other in H10 (Ω). Let E1 = H10 (Ω−)⊕ H10 (Ω0)⊕ H10 (Ω+), and let E2, E3, E4 be the orthogonal complements of E1 in H10 (Ω),H 1 0 (Ω−)⊕H10 (Ω0) in H10 (Ω− ∪Ω0), and H10 (Ω+)⊕H10 (Ω0) in H10 (Ω+ ∪Ω0), respectively. The argument in Proposition 2.1 of [6] shows that E2 is the direct sum of E3 and E4; i.e., E2 = E3 ⊕ E4. We need the following decomposition lemma of H10 (Ω), which comes from [6]. Proposition 2.1. Let Ω be a bounded domain with smooth boundary andΩ = Ω+ ∪Ω0 ∪Ω−. Suppose that Ω− ∩Ω+ = ∅, and the interiors of Ω0,Ω− andΩ+ are nonempty and disjoint from each other. Then we have the following direct decomposition of H10 (Ω): X = H10 (Ω) = X1 ⊕ X2, S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 2833 where X1 = H10 (Ω− ∪Ω0) = H10 (Ω−)⊕ H10 (Ω0)⊕ E3, and X2 = {u ∈ H10 (Ω+ ∪Ω0)|u ⊥ H10 (Ω0)} = H10 (Ω+)⊕ E4. As a consequence of the direct sum and equivalent norm theorem, there is a positive constant C such that C−1 ∫ Ω (|∇v|2 + |∇w|2)dx ≤ ∫ Ω |∇u|2dx ≤ C ∫ Ω (|∇v|2 + |∇w|2)dx, (2.1) where u = v + w, v ∈ X1, w ∈ X2. This inequality is important in order to get the critical groups at zero. Remark 2.2. This decomposition lemma of H10 (Ω) can be extended to that ofW 1,p 0 (Ω). For more details, we refer to [7]. For u = v + w with u ∈ H10 (Ω), v ∈ X1, andw ∈ X2, we have I(u) = 1 2 ∫ Ω (|∇(v + w)|2 − λ(v + w)2)dx− ∫ Ω h−(x)G1(v + w)dx− ∫ Ω h+(x)G2(v + w)dx = 1 2 ∫ Ω (|∇(v + w)|2 − λ(v + w)2)dx− ∫ Ω h−(x)G1(v)dx− ∫ Ω h+(x)G2(w)dx = I(v,w). Then the indefinite nonlinear part of I is separable of variables in v andw; i.e.,∫ Ω h(x)G(u)dx = ∫ Ω h−(x)G1(v)dx+ ∫ Ω h+(x)G2(w)dx. For ε > 0, define E+ε := { u ∈ H10 (Ω) ∣∣∣∣∫ Ω h(x)|u|qdx ≥ qε(‖v‖22 + ‖w‖22), w 6= 0 } . We now study the topological property of E+ε ∩ Bρ , for some ρ > 0, where Bρ is the ball with radius ρ centered at 0. Obviously, E+ε ∪ {0} is star shaped with respect to the origin. Lemma 2.3. For some ρ > 0, the set E+ε ∩ Bρ is contractible in itself. Proof. Let us consider the following deformation on E+ε ∩ Bρ : η(t, u) = tv + w, t ∈ [0, 1]. Clearly, if u ∈ E+ε ∩ Bρ ,∫ Ω h(x)|η(t, u)|qdx = ∫ Ω h+(x)|w|qdx+ tq ∫ Ω h−(x)|v|qdx = ∫ Ω h(x)|u|qdx− (1− tq) ∫ Ω h−(x)|v|qdx ≥ qε(‖v‖22 + ‖w‖22)− (1− tq) ∫ Ω h−(x)|v|qdx ≥ qε(‖tv‖22 + ‖w‖22), for t ∈ [0, 1]. Therefore η : [0, 1]× (E+ε ∩ Bρ)→ E+ε ∩ Bρ . Hence Nρ := { w ∈ X2 ∣∣∣∣∫Ω h+(x)|w|qdx ≥ qε‖w‖22, w 6= 0}∩ Bρ is a strong deformation retract of E+ε ∩ Bρ . Note that ∫ Ω h+(x)|w|qdx > 0,∀w ∈ X2 \ {0}. We can choose ρ > 0, such that, for each w ∈ {X2 \ (Nρ ∪ {0})} ∩ Bρ , there is a unique t(w) = ( ∫ Ω |w|qdx qε ∫ Ω|w|2dx ) 1 2−q ∈ (0, 1] satisfying∫ Ω h+(x)|w|qdx = qε ∫ Ω |w|2dx. Obviously, t(w) is continuous aboutw. Hence we can define a deformation retract η : (X2 \ {0}) ∩ Bρ → (X2 \ {0}) ∩ Bρ : η(w) = { t(w)w, w 6∈ Nρ, w, w ∈ Nρ . 2834 S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 This proves that Nρ ' (X2 \ {0}) ∩ Bρ , since (X2 \ {0}) ∩ Bρ ' Sρ ∩ X2, where Sρ is the sphere with radius ρ. Therefore E+ε ∩ Bρ is contractible in itself. This completes the proof. � Now, we verify that I(u) satisfies a weaker version of the (P.S.) condition on H10 (Ω). Let 0 < λ1 < λ2 ≤ · · · ≤ λk < λ < λk+1 ≤ · · · be a sequence of eigenvalues of −∆ on H10 (Ω) and let e1, e2, . . . be the corresponding orthonormal eigenfunctions in L2(Ω). Define H+(Ω) = span{ek+1, ek+2, . . .}, H−(Ω) = span{e1, . . . , ek}. Then H10 (Ω) = H+(Ω)⊕ H−(Ω). Let P± be the projections (orthogonal) of these subspaces. Set x± = P±x. Let {un} be any sequence satisfying ‖I ′(un)‖ = o(‖un‖) and I(un) ≤ C . Then (I ′(un), u±n ) = ∫ Ω |∇(vn + wn)±|2 − λ((vn + wn)±)2dx− ∫ Ω h−(x)g1(vn)v±n dx− ∫ Ω h+(x)g2(wn)w±n dx ≥ ∫ Ω |∇(vn + wn)±|2 − λ((vn + wn)±)2dx− C ( (‖v±n ‖2) q 2 + (‖w±m‖2) q 2 ) + o(‖v±n ‖22 + ‖w±n ‖22). By (2.1), we get the boundedness of u±n . Arguments as in [6] show that {un} contains a convergent subsequence. Therefore, we have the following proposition. Proposition 2.4. Suppose that C is a constant and that λ 6∈ σ(Ω), un ∈ H10 (Ω) such that I(un) ≤ C, ‖I ′(un)‖ = o(‖un‖) as n→∞. Then {un} is bounded and contains a convergent subsequence. Proposition 2.5. Assume that gi satisfies (g1) and (g2), i = 1, 2. Let h be a continuous function satisfying the ‘‘thick’’ zero set condition (H). λ < λ1(Ω0) and λ 6∈ σ(Ω). Then the set of solutions of (1.1) is bounded. Proof. Noting that (I ′(u), u) = 0, one can obtain that ‖u‖2 ≤ C(1 + ‖u‖22). Therefore, it suffices to prove that the set of critical points of I(u) is bounded in the L2(Ω) norm. For otherwise, we may assume that there exists a sequence of critical points {un} of I(u)with ‖un‖2 →∞. Let u˜n = un‖un‖2 ; then ‖un‖2 ≤ C(1+‖un‖22) implies that the u˜n are bounded in H10 (Ω). Hence, one may assume that u˜n → u0 weakly in H10 (Ω) and strongly in L2(Ω). For any φ ∈ H10 (Ω), 0 = ( I ′(u) ‖un‖2 , φ ) = ∫ Ω ∇u˜n · ∇φ − λu˜nφdx− ∫ Ω h(x) g(un)φ ‖un‖2 dx. Let n→∞; by assumption (g1), one can show that∫ Ω ∇u0 · ∇φ − λu0φdx = 0, which contradicts λ 6∈ σ(Ω). This completes the proof. � It is well known that any nontrivial solution of{−∆u− λu± = h−(x)g±1 (u)+ h+(x)g±2 (u) inΩ, u = 0 on ∂Ω, is a positive (negative) solution of (1.1), where g±i (u) = gi(u) ± u ≥ 0, g±i (u) = 0 ± u ≤ 0, i = 1, 2. We consider the corresponding functional I±(u) = 12 ∫ Ω (|∇u|2 − λu2±)dx− ∫ Ω h−(x)G±1 (v)dx− ∫ Ω h+(x)G±2 (w)dx, where G±i (u) = ∫ u 0 g ± i (s)ds, i = 1, 2. Remark 2.6. In order to compute the critical groups of I±(u) at zero, we consider the following set: E˜+ε := { u ∈ H10 (Ω) ∣∣∣∣∫ Ω h(x)|u+|qdx ≥ qε(‖v+‖22 + ‖w+‖22), w+ 6= 0 } . As in [6], we can show that forw ∈ X2, ∫ Ω h+(x)|w+|qdx > 0 if and only ifw+ 6= 0. Meanwhile, similar arguments as in the proof of Lemma 2.3 show that E˜+ε ∩ Bρ ' { w ∈ X2|qε‖w+‖ ≤ ∫ Ω h+(x)|w+|qdx, w+ 6= 0 } , which is homotopy equivalent to the set S := {w ∈ X2|‖w‖ = ρ andw+ 6= 0}. Since S is contractible in itself (for more details see [6, p. 274, Claim 3]), E˜+ε ∩ Bρ is contractible in itself too. S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 2835 A similar discussion holds for Eˆ+ε := { u ∈ H10 (Ω) ∣∣∣∣∫ Ω h(x)|u−|qdx ≥ qε(‖v−‖22 + ‖w−‖22), w− 6= 0 } . For a ∈ R, let Ia = {u ∈ X |I(u) ≤ a} be the level set of I .We consider the deformationη(t, ·) : (I0\E+ε ∪{0})∩Bρ → I0∩Bρ defined by the formula η(t, u) = tv + w with v ∈ X1 andw ∈ X2. Proposition 2.7. Assume that gi satisfies (g1) and (g2), i = 1, 2. Let h be a continuous function satisfying the ‘‘thick’’ zero set condition (H). λ < λ1(Ω0), λ 6∈ σ(Ω), and 0 < ε < (λ1(Ω0)−λ)2C1 . We have that (i) there is a constant ρ > 0 small enough such that d dt ∣∣∣∣ t=1 I(η(t, u)) > 0, ∀u ∈ {I0 \ (E+ε ∪ {0})} ∩ Bρ; (ii) there exist ρ > 0, and δ > 0 which does not depend on ε, such that∫ Ω h+(x)|w|qdx > δ(‖v‖22 + ‖w‖22), ∀u ∈ {I0 \ (E+ε ∪ {0})} ∩ Bρ . Proof. By assumption, λ+ 2C1ε < λ1(Ω0). We first make a claim. Claim. There exists ρ > 0 small enough, such thatw 6= 0, for u ∈ {I0 \ (E+ε ∪{0})}∩Bρ , with u = v+w, v ∈ X1, andw ∈ X2. For otherwise, ∀ρi → 0, there exists ui ∈ X1 ∩ {I0 \ (E+ε ∪ {0})} ∩ Bρi withwi = 0. Thus 0 ≥ I(ui) = 12 ∫ Ω (|∇ui|2 − λu2i )dx− 1 q C1 ∫ Ω h−(x)|ui|qdx+ o(‖ui‖22) ≥ 1 2 ∫ Ω (|∇ui|2 − λu2i )dx− εC1 ∫ Ω |ui|2dx+ o(‖ui‖22), (2.2) which implies that ∫ Ω |∇ui|2 ≤ C ∫ Ω u2i dx. Let u˜i = ui‖ui‖2 . We infer u˜i is bounded in H10 (Ω). So we may assume that u˜i → u0 weakly in H10 (Ω); u˜i → u0 strongly in L2(Ω). Note that u0 6= 0 as ‖u0‖2 = 1. Thus 0 ≥ 1 2 (‖u˜i‖2 − λ)− C1 q‖ui‖2−q2 ∫ Ω h−(x)|u˜i|qdx+ o(1), which implies that − 1‖ui‖2−q2 ∫ Ω h−(x)|u˜i|qdx ≤ C . However, 1‖ui‖2−q2 → +∞, as ρi → 0. We get ∫ Ω h−(x)|u˜i|qdx → ∫ Ω h−(x)|u0|qdx = 0. Therefore, u0(x) = 0 on Ω−. Since Supp u0 ⊂ cl(Ω0 ∩Ω−), we obtain u0 ∈ H10 (Ω0). Dividing (2.2) by ‖ui‖22, we have 0 ≥ ∫ Ω (|∇u˜i|2 − (λ+ 2C1ε)|u˜i|2)dx+ o(1). By the weak lower continuity of ‖ · ‖, 0 ≥ ∫ Ω |∇u0|2dx− (λ+ 2C1ε), with ‖u0‖2 = 1 which contradicts λ+ 2C1ε < λ1(Ω0). This proves the claim. Note that η(t, u) = tv + w, where v ∈ X1,w ∈ X2, and I(η(t, u)) = 1 2 ∫ Ω (|∇(tv + w)|2 − λ(tv + w)2)dx− ∫ Ω h−(x)G1(tv)dx− ∫ Ω h+(x)G2(w)dx. (2.3) 2836 S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 Then d dt I(η(t, u)) = ∫ Ω (∇(tv + w) · ∇v − λ(tv + w)v)dx− ∫ Ω h−(x)g1(tv)vdx = t ∫ Ω (|∇v|2 − λv2)dx+ ∫ Ω (∇v · ∇w − λvw)dx− ∫ Ω h−(x)g1(tv)vdx. (2.4) Recalling that u 6∈ E+ε , thus− ∫ Ω h(x)|u|qdx ≥ −qε(‖v‖22 + ‖w‖22). It follows that 0 ≥ 1 2 ∫ Ω |∇u|2 − λu2dx− ∫ Ω h(x)G(u)dx = 1 2 ∫ Ω |∇u|2 − λu2dx− 1 q C1 ∫ Ω h(x)|u|qdx+ o(‖v‖22 + ‖w‖22) ≥ 1 2 ∫ Ω |∇u|2 − λu2dx− C1ε(‖v‖22 + ‖w‖22)+ o(‖v‖22 + ‖w‖22). (2.5) By (2.1), we obtain C−1 ∫ Ω |∇v|2 + |∇w|2dx ≤ ∫ Ω |∇u|2dx ≤ 2λ ∫ Ω (v + w)2dx+ 2C1ε ∫ Ω v2 + w2dx+ o(‖vn‖22 + ‖wn‖22) ≤ C ∫ Ω v2 + w2dx. (2.6) If (i) of the proposition is false, then there exists un ∈ I0 \ (E+ε ∪ {0})with ‖un‖ → 0 such that ddt ∣∣∣∣ t=1 I(η(t, un)) ≤ 0. By the claim above, ‖vn‖2 + ‖wn‖2 6= 0. Therefore we can define u˜n = un‖vn‖2 + ‖wn‖2 , v˜n = vn ‖vn‖2 + ‖wn‖2 , w˜n = wn ‖vn‖2 + ‖wn‖2 . Since ‖un‖ → 0, (2.6) implies that∫ Ω (|∇v˜n|2 + |∇w˜n|2)dx ≤ C . As a consequence, we may assume v˜n → v0, w˜n → w0, u˜n → u0 weakly in H10 (Ω), v˜n → v0, w˜n → w0, u˜n → u0 strongly in L2(Ω), with u0 = v0 + w0. Note that u0 6= 0 as ‖v0‖2 + ‖w0‖2 = 1. By (2.4), we have 0 ≥ d dt ∣∣∣∣ t=1 I(η(t, un)) = ∫ Ω (|∇vn|2 − λ|vn|2)dx+ ∫ Ω (∇vn · ∇wn − λvnwn)dx− C1 ∫ Ω h−(x)|vn|qdx+ o(‖vn‖22 + ‖wn‖22). Dividing by (‖vn‖2 + ‖wn‖2)2, we get 0 ≥ d dt ∣∣∣∣ t=1 I(η(t, un)) (‖vn‖2 + ‖wn‖2)2 = ∫ Ω (|∇v˜n|2 − λ|v˜n|2)dx+ ∫ Ω (∇v˜n · ∇w˜n − λv˜nw˜n)dx− C1 ∫ Ω h−(x)|vn|q (‖vn‖2 + ‖wn‖2)2 dx+ o(‖v˜n‖ 2 2 + ‖w˜n‖22). (2.7) It follows that − 1 (‖vn‖2 + ‖wn‖2)2−q ∫ Ω h−(x)|v˜n|qdx ≤ C . Therefore, we deduce that∫ Ω h−(x)|v˜n|qdx→ ∫ Ω h−(x)|v0|qdx = 0. S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 2837 Thus v0(x) = 0,∀x ∈ Ω−. Recalling that v0 ∈ X1, one has v0 ∈ H10 (Ω0). Since un 6∈ E+ε , it follows that 1 (‖vn‖2 + ‖wn‖2)2−q ∫ Ω h+(x)|w˜n|qdx ≤ − 1 (‖vn‖2 + ‖wn‖2)2−q ∫ Ω h−(x)|v˜n|qdx+ qε(‖v˜n‖22 + ‖w˜n‖22) ≤ C . Therefore,∫ Ω h+(x)|w˜n|qdx→ ∫ Ω h+(x)|w0|qdx = 0, which means thatw0(x) = 0,∀x ∈ Ω+. Hencew0 = 0, and ‖v0‖2 = 1. Now, we decomposeH10 (Ω) again. Set Y − := H10 (Ω−)⊕E3, Y+ := H10 (Ω+)⊕E4 and Y := Y−⊕Y+. Let Z := H10 (Ω0). We write un = yn + zn = y−n + y+n + zn with y±n ∈ Y± and zn ∈ Z . Obviouslywn = y+n , vn = y−n + zn. Define y˜±n := y ± n ‖vn‖2+‖wn‖2 , y˜n = y˜−n + y˜+n , and z˜n := zn‖vn‖2+‖wn‖2 . Invoking (2.6), ‖y˜±n ‖, ‖z˜n‖ are bounded. Then we may assume that y˜±n → y±0 and z˜n → z0 weakly in H10 (Ω), and strongly in L2(Ω). The above arguments imply that y±0 = 0 and ‖z0‖2 = 1. Obviously, v0 = z0. It follows from un 6∈ E+ε that−qε ∫ Ω (|y˜−n |2 + |z˜n|2 + 2y˜−n z˜n + |y˜+n |2)dx ≤ − ∫ Ω h(x)|u|q (‖vn‖2+‖wn‖2)2 dx. Noting that λ+ 2C1ε < λ1(Ω0), then ‖z˜n‖ ≥ √λ+ 2C1ε‖z˜n‖2. By (2.5), we have∫ Ω |∇ y˜n|2 − λ|y˜n|2dx ≤ ∫ Ω |∇ y˜n|2 − λ|y˜n|2dx+ ∫ Ω |∇ z˜n|2 − (λ+ 2C1ε)|z˜n|2dx = ∫ Ω |∇u˜n|2 − λ|u˜n|2dx+ 2λ ∫ Ω y˜nz˜ndx− 2C1ε ∫ Ω |z˜n|2dx ≤ ∫ Ω |∇u˜n|2 − λ|u˜n|2dx+ 2λ ∫ Ω y˜nz˜ndx− 2C1q ∫ Ω h(x)|u|q (‖vn‖2 + ‖wn‖2)2 dx+ 2C1ε ∫ Ω (|y˜−n |2 + 2y˜−n z˜n + |y˜+n |2)dx ≤ 2I(un) (‖vn‖2 + ‖wn‖2)2 + 2λ ∫ Ω y˜nz˜ndx+ 2C1ε ∫ Ω (|y˜−n |2 + 2y˜−n z˜n + |y˜+n |2)dx+ o(‖v˜n‖22 + ‖w˜n‖22) ≤ 2λ ∫ Ω y˜nz˜ndx+ 2C1ε ∫ Ω (|y˜−n |2 + 2y˜−n z˜n + |y˜+n |2)dx+ o(‖v˜n‖22 + ‖w˜n‖22). It follows that lim supn→∞ ‖y˜n‖2 ≤ lim infn→∞ ( λ‖y˜n‖22 + 2C1ε ∫ Ω (|y˜−n |2 + 2y˜−n z˜n + |y˜+n |2)dx+ 2λ ∫ Ω y˜nz˜ndx ) = 0. Hence we conclude that y˜n → 0 strongly in H10 (Ω). Since un ∈ {I0 \ (E+ε ∪ {0})} ∩ Bρ , we establish 0 ≥ 2I(un) (‖vn‖2 + ‖wn‖2)2 ≥ ∫ Ω |∇ y˜n|2 − λ|y˜n|2dx+ ∫ Ω |∇ z˜n|2 − λ|z˜n|2dx − 2λ ∫ Ω y˜nz˜ndx− 2C1q ∫ Ω h(x)|un|qdx (‖vn‖2 + ‖wn‖2)2 dx+ o(‖v˜n‖ 2 2 + ‖w˜n‖22) ≥ ∫ Ω |∇ y˜n|2 − λ|y˜n|2dx+ ∫ Ω |∇ z˜n|2 − λ|z˜n|2dx− 2λ ∫ Ω y˜nz˜ndx− 2C1ε(‖v˜n‖22 + ‖w˜n‖22)+ o(‖v˜n‖22 + ‖w˜n‖22). Let n→∞; by the weak lower continuity of ‖ · ‖, we have 0 ≥ ∫ Ω0 |∇z0|2 − (λ+ 2C1ε)|z0|2dx, with ‖z0‖2 = 1, which contradicts λ+ 2C1ε < λ1(Ω0). This completes the proof of (i). The proof of (ii) is the same as that of (i). We only sketch the differences. If (ii) is not true, then for any δn → 0, there exists un ∈ I0 \ (E+ε ∪ {0})with ‖un‖ → 0, such that∫ Ω h+(x)|wn|qdx ≤ δn(‖vn‖22 + ‖wn‖22). (2.8) It suffices to replace (2.7) by the following arguments. Noting that un ∈ I0, by (2.8), 0 ≥ 1 2 ∫ Ω (|∇vn|2 + |∇wn|2 + 2∇vn · ∇wn)dx− λ2 ∫ Ω (|vn|2 + |wn|2 + 2vnwn)dx − C1δn q (‖vn‖22 + ‖wn‖22)− C1 q ∫ Ω h−(x)|vn|qdx+ o(‖vn‖22 + ‖wn‖22). 2838 S. Liu / Nonlinear Analysis 73 (2010) 2831–2841 Dividing by (‖vn‖2 + ‖wn‖2)2, we deduce that 0 ≥ 1 2 ∫ Ω (|∇v˜n|2 + |∇w˜n|2 + 2∇v˜n · ∇w˜n)dx− λ2 ∫ Ω (|v˜n|2 + |w˜n|2 + 2v˜nw˜n)dx − C1δn q (‖v˜n‖22 + ‖w˜n‖22)− C1 q(‖vn‖2 + ‖wn‖2)2−q ∫ Ω h−(x)|v˜n|qdx+ o(‖v˜n‖22 + ‖w˜n‖22), which implies that − 1 (‖vn‖2 + ‖wn‖2)2−q ∫ Ω h−(x)|v˜n|qdx ≤ C . Similar arguments as those in (i) lead to a contradiction, which completes the proof. � Proposition 2.8. Assume that gi satisfies (g1) and (g2), i = 1, 2. Let h be a continuous function satisfying the ‘‘thick’’ zero set condition (H). Then there exists ρ > 0, such that d dt ∣∣∣∣ t=1 I(tu) > 0 ∀u ∈ Mρ := {u ∈ E+ε ∩ Bρ |I(u) ≥ 0}. Proof. Noting that u 6= 0 and the assumptions, we have 1 2 d dt ∣∣∣∣ t=1 I(tu) ≥ 1 2 ∫ Ω (|∇u|2 − λu2)dx− C1 2 ∫ Ω h(x)|u|qdx+ o(‖v‖22 + ‖w‖22) = I(u)+ ( 1 q − 1 2 ) C1 ∫ Ω h(x)|u|qdx+ o(‖v‖22 + ‖w‖22) ≥ I(u)+ ( 1 q − 1 2 ) C1qε(‖v‖22 + ‖w‖22)+ o(‖v‖22 + ‖w‖22). One concludes that