Pumping lemma for regular languages

In the theory of formal languages, the pumping lemma for regular languages is a lemma that describes an essential property of all regular languages. Informally, it says that all sufficiently long words in a regular language may be pumped—that is, have a middle section of the word repeated an arbitrary number of times—to produce a new word that also lies within the same language.

Specifically, the pumping lemma says that for any regular language ${\displaystyle L}$ there exists a constant ${\displaystyle p}$ such that any word ${\displaystyle w}$ in ${\displaystyle L}$ with length at least ${\displaystyle p}$ can be split into three substrings, ${\displaystyle w=xyz}$, where the middle portion ${\displaystyle y}$ must not be empty, such that the words ${\displaystyle xz,xyz,xyyz,xyyyz,...}$ constructed by repeating ${\displaystyle y}$ zero or more times are still in ${\displaystyle L}$. This process of repetition is known as "pumping". Moreover, the pumping lemma guarantees that the length of ${\displaystyle xy}$ will be at most ${\displaystyle p}$, imposing a limit on the ways in which ${\displaystyle w}$ may be split. Finite languages vacuously satisfy the pumping lemma by having ${\displaystyle p}$ equal to the maximum string length in ${\displaystyle L}$ plus one.

The pumping lemma is useful for disproving the regularity of a specific language in question. It was first proven by Michael Rabin and Dana Scott in 1959,[1] and rediscovered shortly after by Yehoshua Bar-Hillel, Micha A. Perles, and Eli Shamir in 1961, as a simplification of their pumping lemma for context-free languages.[2][3]

Formal statement

Let ${\displaystyle L}$ be a regular language. Then there exists an integer ${\displaystyle p\geq 1}$ depending only on ${\displaystyle L}$ such that every string ${\displaystyle w}$ in ${\displaystyle L}$ of length at least ${\displaystyle p}$ (${\displaystyle p}$ is called the "pumping length"[4]) can be written as ${\displaystyle w=xyz}$ (i.e., ${\displaystyle w}$ can be divided into three substrings), satisfying the following conditions:

• ${\displaystyle |y|\geq 1}$
• ${\displaystyle |xy|\leq p}$
• ${\displaystyle (\forall n\geq 0)(xy^{n}z\in L)}$

${\displaystyle y}$ is the substring that can be pumped (removed or repeated any number of times, and the resulting string is always in ${\displaystyle L}$). (1) means the loop ${\displaystyle y}$ to be pumped must be of length at least one; (2) means the loop must occur within the first ${\displaystyle p}$ characters. ${\displaystyle |x|}$ must be smaller than ${\displaystyle p}$ (conclusion of (1) and (2)), but apart from that, there is no restriction on ${\displaystyle x}$ and ${\displaystyle z}$.

In simple words, for any regular language ${\displaystyle L}$, any sufficiently long word ${\displaystyle w}$ (in ${\displaystyle L}$) can be split into 3 parts. i.e. ${\displaystyle w=xyz}$ , such that all the strings ${\displaystyle xy^{n}z}$ for ${\displaystyle n\geq 0}$ are also in ${\displaystyle L}$.

Below is a formal expression of the Pumping Lemma.

${\displaystyle {\begin{array}{l}(\forall L\subseteq \Sigma ^{*})\\\quad ({\mbox{regular}}(L)\Rightarrow \\\quad ((\exists p\geq 1)((\forall w\in L)((|w|\geq p)\Rightarrow \\\quad ((\exists x,y,z\in \Sigma ^{*})(w=xyz\land (|y|\geq 1\land |xy|\leq p\land (\forall n\geq 0)(xy^{n}z\in L))))))))\end{array}}}$

Use of the lemma

The pumping lemma is often used to prove that a particular language is non-regular: a proof by contradiction (of the language's regularity) may consist of exhibiting a word (of the required length) in the language that lacks the property outlined in the pumping lemma.

For example, the language ${\displaystyle L=\{a^{n}b^{n}:n\geq 0\}}$ over the alphabet ${\displaystyle \Sigma =\{a,b\}}$ can be shown to be non-regular as follows:

Let ${\displaystyle w,x,y,z,p}$, and ${\displaystyle n}$ be as used in the formal statement for the pumping lemma above. We assume that there exists some constant ${\displaystyle p}$. Let ${\displaystyle w}$ in ${\displaystyle L}$ be given by ${\displaystyle w=a^{p}b^{p}}$, which is a string longer than ${\displaystyle p}$. By the pumping lemma, there must be some decomposition ${\displaystyle w=xyz}$ with ${\displaystyle |xy|\leq p}$ and ${\displaystyle |y|\geq 1}$ such that ${\displaystyle xy^{i}z}$ in ${\displaystyle L}$ for every ${\displaystyle i\geq 0}$. Using ${\displaystyle |xy|\leq p}$, we know ${\displaystyle y}$ only consists of instances of ${\displaystyle a}$. Moreover, because ${\displaystyle |y|\geq 1}$, it contains at least one instance of the letter ${\displaystyle a}$. We now pump ${\displaystyle y}$ up: ${\displaystyle xy^{2}z}$ has more instances of the letter ${\displaystyle a}$ than the letter ${\displaystyle b}$, since we have added some instances of ${\displaystyle a}$ without adding instances of ${\displaystyle b}$. Therefore, ${\displaystyle xy^{2}z}$ is not in ${\displaystyle L}$. We have reached a contradiction. Therefore, the assumption that ${\displaystyle L}$ is regular (i.e. there exists such a ${\displaystyle p}$) must be incorrect. Hence ${\displaystyle L}$ is not regular.

The proof that the language of balanced (i.e., properly nested) parentheses is not regular follows the same idea. Given ${\displaystyle p}$, there is a string of balanced parentheses that begins with more than ${\displaystyle p}$ left parentheses, so that ${\displaystyle y}$ will consist entirely of left parentheses. By repeating ${\displaystyle y}$, we can produce a string that does not contain the same number of left and right parentheses, and so they cannot be balanced.

Proof of the pumping lemma

Proof idea: Whenever a sufficiently long string xyz is recognized by a finite automaton, it must have reached some state (${\displaystyle q_{s}=q_{t}}$) twice. Hence, after repeating ("pumping") the middle part ${\displaystyle y}$ arbitrarily often (xyyz, xyyyz, ...) the word will still be recognized.

For every regular language there is a finite state automaton (FSA) that accepts the language. The number of states in such an FSA are counted and that count is used as the pumping length ${\displaystyle p}$. For a string of length at least ${\displaystyle p}$, let ${\displaystyle q_{0}}$ be the start state and let ${\displaystyle q_{1},...,q_{p}}$ be the sequence of the next ${\displaystyle p}$ states visited as the string is emitted. Because the FSA has only ${\displaystyle p}$ states, within this sequence of ${\displaystyle p+1}$ visited states there must be at least one state that is repeated. Write ${\displaystyle q_{s}}$ for such a state. The transitions that take the machine from the first encounter of state ${\displaystyle q_{s}}$ to the second encounter of state ${\displaystyle q_{s}}$ match some string. This string is called ${\displaystyle y}$ in the lemma, and since the machine will match a string without the ${\displaystyle y}$ portion, or with the string ${\displaystyle y}$ repeated any number of times, the conditions of the lemma are satisfied.

For example, the following image shows an FSA.

The FSA accepts the string: abcd. Since this string has a length at least as large as the number of states, which is four, the pigeonhole principle indicates that there must be at least one repeated state among the start state and the next four visited states. In this example, only ${\displaystyle q_{1}}$ is a repeated state. Since the substring bc takes the machine through transitions that start at state ${\displaystyle q_{1}}$ and end at state ${\displaystyle q_{1}}$, that portion could be repeated and the FSA would still accept, giving the string abcbcd. Alternatively, the bc portion could be removed and the FSA would still accept giving the string ad. In terms of the pumping lemma, the string abcd is broken into an ${\displaystyle x}$ portion a, a ${\displaystyle y}$ portion bc and a ${\displaystyle z}$ portion d.

General version of pumping lemma for regular languages

If a language ${\displaystyle L}$ is regular, then there exists a number ${\displaystyle p\geq 1}$ (the pumping length) such that every string ${\displaystyle uwv}$ in ${\displaystyle L}$ with ${\displaystyle |w|\geq p}$ can be written in the form

${\displaystyle uwv=uxyzv}$

with strings ${\displaystyle x}$, ${\displaystyle y}$ and ${\displaystyle z}$ such that ${\displaystyle |xy|\leq p}$, ${\displaystyle |y|\geq 1}$ and

${\displaystyle uxy^{i}zv}$ is in ${\displaystyle L}$ for every integer ${\displaystyle i\geq 0}$.[5]

From this, the above standard version follows a special case, with both ${\displaystyle u}$ and ${\displaystyle v}$ being the empty string.

Since the general version imposes stricter requirements on the language, it can be used to prove the non-regularity of many more languages, such as ${\displaystyle \{a^{m}b^{n}c^{n}:m\geq 1{\text{ and }}n\geq 1\}}$.[6]

Converse of lemma not true

While the pumping lemma states that all regular languages satisfy the conditions described above, the converse of this statement is not true: a language that satisfies these conditions may still be non-regular. In other words, both the original and the general version of the pumping lemma give a necessary but not sufficient condition for a language to be regular.

For example, consider the following language:

${\displaystyle {\begin{matrix}L&=&\{uvwxy:u,y\in \{0,1,2,3\}^{*};v,w,x\in \{0,1,2,3\}\land (v=w\lor v=x\lor x=w)\}\\&&\cup \ \{w:w\in \{0,1,2,3\}^{*}\land {\text{precisely }}{\tfrac {1}{7}}{\text{ of the characters in }}w{\text{ are 3's}}\}\end{matrix}}}$.

In other words, ${\displaystyle L}$ contains all strings over the alphabet ${\displaystyle \{0,1,2,3\}}$ with a substring of length 3 including a duplicate character, as well as all strings over this alphabet where precisely 1/7 of the string's characters are 3's. This language is not regular but can still be "pumped" with ${\displaystyle p=5}$. Suppose some string s has length at least 5. Then, since the alphabet has only four characters, at least two of the first five characters in the string must be duplicates. They are separated by at most three characters.

• If the duplicate characters are separated by 0 characters, or 1, pump one of the other two characters in the string, which will not affect the substring containing the duplicates.
• If the duplicate characters are separated by 2 or 3 characters, pump 2 of the characters separating them. Pumping either down or up results in the creation of a substring of size 3 that contains 2 duplicate characters.
• The second condition of ${\displaystyle L}$ ensures that ${\displaystyle L}$ is not regular: Consider the string ${\displaystyle (013)^{3m}(012)^{i}}$. This string is in ${\displaystyle L}$ exactly when ${\displaystyle i=4m}$ and thus ${\displaystyle L}$ is not regular by the Myhill–Nerode theorem.

The Myhill–Nerode theorem provides a test that exactly characterizes regular languages. The typical method for proving that a language is regular is to construct either a finite state machine or a regular expression for the language.

See also

• Pumping lemma for context-free languages
• Ogden's lemma

Notes

1. Rabin, Michael; Scott, Dana (Apr 1959). "Finite Automata and Their Decision Problems" (PDF). IBM Journal of Research and Development. 3 (2): 114–125. doi:10.1147/rd.32.0114. Archived from the original on December 14, 2010.CS1 maint: unfit URL (link) Here: Lemma 8, p.119
2. Bar-Hillel, Y.; Perles, M.; Shamir, E. (1961), "On formal properties of simple phrase structure grammars", Zeitschrift für Phonetik, Sprachwissenschaft und Kommunikationsforschung, 14 (2): 143–172
3. John E. Hopcroft; Rajeev Motwani; Jeffrey D. Ullman (2003). Introduction to Automata Theory, Languages, and Computation. Addison Wesley. Here: Sect.4.6, p.166
4. Berstel, Jean; Lauve, Aaron; Reutenauer, Christophe; Saliola, Franco V. (2009). Combinatorics on words. Christoffel words and repetitions in words. CRM Monograph Series. 27. Providence, RI: American Mathematical Society. p. 86. ISBN 978-0-8218-4480-9. Zbl 1161.68043.
5. Savitch, Walter (1982). Abstract Machines and Grammars. p. 49. ISBN 978-0-316-77161-0.
6. John E. Hopcroft & Jeffrey D. Ullman (1979). Introduction to Automata Theory, Languages, and Computation. Reading/MA: Addison-Wesley. ISBN 978-0-201-02988-8. Here: p. 72, Exercise 3.2 (which give a slightly less general version, requiring |w|=p) and 3.3

References

• Lawson, Mark V. (2004). Finite automata. Chapman and Hall/CRC. ISBN 978-1-58488-255-8. Zbl 1086.68074.
• Sipser, Michael (1997). "1.4: Nonregular Languages". Introduction to the Theory of Computation. PWS Publishing. pp. 77–83. ISBN 978-0-534-94728-6. Zbl 1169.68300.
• Hopcroft, John E.; Ullman, Jeffrey D. (1979). Introduction to Automata Theory, Languages, and Computation. Reading, Massachusetts: Addison-Wesley Publishing. ISBN 978-0-201-02988-8. Zbl 0426.68001. (See chapter 3.)
• Bakhadyr Khoussainov; Anil Nerode (6 December 2012). Automata Theory and its Applications. Springer Science & Business Media. ISBN 978-1-4612-0171-7.