02-ruby-basics/02.05-regular-expression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正则表达式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[正则表达式](https://en.wikipedia.org/wiki/Regular_expression)是用来匹配字符串或者子串的一种模式，匹配的字符串可以很具体，也可以很一般化。\n",
    "\n",
    "**Ruby** 内建支持了正则表达式。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 创建 `Regexp`\n",
    "\n",
    "### 典型构造\n",
    "\n",
    "典型地，**Ruby** 中可以使用一对 `/` 生成 `Regexp`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(?-mix:^https?:\\/\\/)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Regexp"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /^https?:\\/\\//\n",
    "puts re\n",
    "re.class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特殊构造\n",
    "\n",
    "* %r: 允许内插值(`interpolation`)\n",
    "* 你可以使用任何典型的分割字符，例如%r{...}, %r(...), %r|...|, %r?...?, %r-....-, etc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(?-mix:^https?:\\/\\/)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Regexp"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = %r(^https?://)\n",
    "puts re\n",
    "re.class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则表达式规则"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "正则表达式由一些普通字符和一些元字符（metacharacters）组成。普通字符包括大小写的字母和数字，而元字符则具有特殊的含义：\n",
    "\n",
    "子表达式|匹配内容\n",
    "---|---\n",
    "`.`| 匹配除了换行符之外的内容\n",
    "`\\w` | 匹配所有字母和数字字符\n",
    "`\\d` | 匹配所有数字，相当于 `[0-9]`\n",
    "`\\s` | 匹配空白，相当于 `[\\t\\n\\t\\f\\v]`\n",
    "`\\W,\\D,\\S`| 匹配对应小写字母形式的补\n",
    "`[...]` | 表示可以匹配的集合，支持范围表示如 `a-z`, `0-9` 等\n",
    "`(...)` | 表示作为一个整体进行匹配\n",
    "&#166; | 表示逻辑或\n",
    "`^` | 表示匹配后面的子表达式的补\n",
    "`*` | 表示匹配前面的子表达式 0 次或更多次\n",
    "`+` | 表示匹配前面的子表达式 1 次或更多次\n",
    "`?` | 表示匹配前面的子表达式 0 次或 1 次\n",
    "`{m}` | 表示匹配前面的子表达式 m 次\n",
    "`{m,}` | 表示匹配前面的子表达式至少 m 次\n",
    "`{m,n}` | 表示匹配前面的子表达式至少 m 次，至多 n 次\n",
    "\n",
    "\n",
    "锚点\n",
    "\n",
    "表达式|匹配内容\n",
    "--- |:---\n",
    "`^`  | 匹配行首\n",
    "`$`  | 匹配行尾\n",
    "`\\A` | 匹配字符串开始\n",
    "`\\Z` | 匹配字符串结束，忽略最后的换行符\n",
    "`\\z` | 匹配字符串结束\n",
    "`\\G` | 匹配第一次匹配的位置\n",
    "`\\b` | 匹配单词边界\n",
    "`\\B` | 匹配非单词边界\n",
    "`(exp)` | 匹配 `exp` , 并捕获文本到自动命名的组里\n",
    "`(?<name>exp)` | 匹配 `exp`，并捕获文本到名称为 `name` 的组里\n",
    "`(?:exp)` | 匹配 `exp` ,不捕获匹配的文本，也不给此分组分配组号\n",
    "`(?=exp)`  | 零宽度正预测先行断言，它断言自身出现的位置的后面能匹配表达式 `exp`\n",
    "`(?<=exp)` | 零宽度正回顾后发断言，它断言自身出现的位置的前面能匹配表达式 `exp`\n",
    "`(?!exp)`  | 零宽度负预测先行断言，断言此位置的后面不能匹配表达式 `exp`\n",
    "`(?<!exp)` | 零宽度负回顾后发断言，断言此位置的前面不能匹配表达式 `exp`\n",
    "\n",
    "\n",
    "\n",
    "例如：\n",
    "\n",
    "- `ca*t       匹配： ct, cat, caaaat, ...`\n",
    "- `ab\\d|ac\\d  匹配： ab1, ac9, ...`\n",
    "- `([^a-q]bd) 匹配： rbd, 5bd, ...`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 简单操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 匹配\n",
    "\n",
    "\n",
    "#### `re === str` 判断 `str` 中是否有符合 `re` 模式中的子串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "true\n",
      "abc is LowerCase\n",
      "DEF is UpperCase\n",
      "123 is Number\n",
      " is Empty\n",
      "aBc is Mixed\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[\"abc\", \"DEF\", \"123\", \"\", \"aBc\"]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "puts /[A-Z]*/ === :aBc\n",
    "\n",
    "['abc', 'DEF', '123', '', 'aBc'].each do|str|\n",
    "  category = case str\n",
    "    when /^[A-Z]+$/; 'UpperCase'\n",
    "    when /^[a-z]+$/; 'LowerCase'\n",
    "    when /^\\d+$/; 'Number'\n",
    "    when /^$/; 'Empty'\n",
    "  else\n",
    "    'Mixed'\n",
    "  end\n",
    "  puts \"#{str} is #{category}\"\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `str =~ re` 和 `re =~ str` 判断 str 中是否有符合 re 模式中的子串，并返回 `re` 模式首次在 `str` 中出现的位置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "/(?<vowel>[aeiou]+)/"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /(?<vowel>[aeiou]+)/ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re =~ 'hello'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'hello' =~ re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nil\n"
     ]
    }
   ],
   "source": [
    "p re =~ 'https'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nil\n"
     ]
    }
   ],
   "source": [
    "p 'https' =~ re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过 `=~` 可以获取到匹配的位置，如果需要知道更多的匹配信息，可以通过内置的全局变量来获取。\n",
    "\n",
    "表达式 | 含义 \n",
    "--- | ---\n",
    "`$~` | 等价于 `RegExp::last_match`，返回最近匹配的 `MatchData`\n",
    "`$&` | 返回最近匹配的完整文本\n",
    "$&#96; | 返回匹配文本前的所有文本\n",
    "`$'` | 返回匹配文本后的所有文本\n",
    "`$1`, `$2`, ... , `$n` | 按照分组编号得到对应的文本\n",
    "`$+` | 返回最后捕获分组的文本"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "/(?<vowel>[aeiou]{2,})/"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /(?<vowel>[aeiou]{2,})/ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "$& = oo\n",
      "$` = this is g\n",
      "$' = d deed\n",
      "$1 = oo, $+=oo\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "#<MatchData \"oo\" vowel:\"oo\">"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "puts 'this is good deed' =~ re\n",
    "\n",
    "puts \"$& = #{$&}\"\n",
    "puts \"$` = #{$`}\"\n",
    "puts \"$' = #{$'}\"\n",
    "puts \"$1 = #{$1}, $+=#{$+}\"\n",
    "$~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n",
      "$& = oo\n",
      "$` = this is g\n",
      "$' = d deed\n",
      "$1 = oo, $+=oo\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "#<MatchData \"oo\" vowel:\"oo\">"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "puts re =~ 'this is good deed'\n",
    "\n",
    "puts \"$& = #{$&}\"\n",
    "puts \"$` = #{$`}\"\n",
    "puts \"$' = #{$'}\"\n",
    "puts \"$1 = #{$1}, $+=#{$+}\"\n",
    "$~"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `match(str[, pos])`\n",
    "\n",
    "在 `Regexp` 实例中， 如果需要获取更多的匹配信息，`match` 是更常用的方法：\n",
    "\n",
    "```ruby\n",
    "re.match(str[, pos])\n",
    "```\n",
    "\n",
    "寻找第一个匹配成功的部分，成功则返回一个 `MatchData` 对象，不成功则返回 `nil`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "#<MatchData \"good\" vowel:\"oo\">"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /[^aeiou](?<vowel>[aeiou]{2,})[^aeiou]/ \n",
    "str = 'this is a good deed in the beautiful world'\n",
    "\n",
    "m = re.match(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MatchText is 'good' with groups {\"vowel\"=>\"oo\"}\n",
      "this is a {g(oo)d}\n",
      "MatchText is 'deed' with groups {\"vowel\"=>\"ee\"}\n",
      " {d(ee)d}\n",
      "MatchText is 'beaut' with groups {\"vowel\"=>\"eau\"}\n",
      " in the {b(eau)t}\n"
     ]
    }
   ],
   "source": [
    "pos = 0\n",
    "while m = re.match(str, pos)\n",
    "  b0, b1, e0, e1 = m.begin(0), m.begin(1), m.end(0), m.end(1)\n",
    "  puts \"MatchText is '#{m.to_s}' with groups #{m.named_captures}\"\n",
    "  puts \"#{str[pos...b0]}{#{str[b0...b1]}(#{str[b1...e1]})#{str[e1...e0]}}\"\n",
    "  pos = e0\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 更多方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 子串\n",
    "\n",
    "`str[regexp]` 和 `str[regexp, capture]` 可以快速定位 `str` 满足 `regexp` 模式的子串。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"good\""
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /[^aeiou](?<vowel>[aeiou]{2,})[^aeiou]/ \n",
    "str = 'this is a good deed in the beautiful world'\n",
    "\n",
    "str[re]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"oo\""
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "str[re, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"oo\""
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "str[re, \"vowel\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Gray, James\""
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"Gray, James\"[/(\\w+), (\\w+)/]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"James\""
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"Gray, James\"[/(\\w+), (\\w+)/, 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分割\n",
    "\n",
    "`str.split(pattern[, limit])` 使用指定的 `pattern` 对 `str` 进行分割。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"1\", \"2.34\", \"56\", \"7\"]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"1, 2.34,56, 7\".split(%r{,\\s*})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 查找\n",
    "\n",
    "`str.scan(pattern)` 查找字符串 `str` 中匹配 `pattern` 的子串"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"this\", \"is\", \"a\", \"good\", \"deed\", \"in\", \"the\", \"beautiful\", \"world\"]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "str = 'this is a good deed in the beautiful world'\n",
    "\n",
    "str.scan(/\\w+/).to_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "good\n",
      "deed\n",
      "beaut\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"this is a good deed in the beautiful world\""
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /(?<g1>[^aeiou])(?<g2>[aeiou]{2,})(?<g3>[^aeiou])/ \n",
    "\n",
    "str.scan(re) do|g1, g2, g3|\n",
    "  puts \"#{g1}#{g2}#{g3}\"\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`array.grep(pattern)` 查找数组 `array` 中匹配 `pattern` 的元素\n",
    "\n",
    "`array.grep_v(pattern)` 查找数组 `array` 中不匹配 `pattern` 的元素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"foo\"]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array = %w[foo bar baz]\n",
    "\n",
    "array.grep(/\\Af/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"bar\", \"baz\"]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "array.grep_v(/\\Af/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 替换\n",
    "\n",
    "`str.gsub(pattern, replacement)` 将字符串 `str` 中匹配 `pattern` 的部分替换成想要的部分 `replacement`，并返回新的字符串。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"this is a **** **** in the *****iful world\""
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re = /[^aeiou](?<vowel>[aeiou]{2,})[^aeiou]/ \n",
    "str = 'this is a good deed in the beautiful world'\n",
    "\n",
    "str.gsub(re) do|m|\n",
    "  '*' * m.length\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"this is a *oo* *ee* in the *eau*iful world\""
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "str.gsub(re, '*\\k<vowel>*')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 更多资料\n",
    "\n",
    "* <http://rubular.com> is a great interactive Ruby regexp toy\n",
    "* [the Pickaxe book](http://www.ruby-doc.org/docs/ProgrammingRuby/html/language.html#UJ) has a just-the-facts-maam overview of regexps\n",
    "* [`cheat regexp`](http://cheat.errtheblog.com/s/regexp) and [`cheat regex`](http://cheat.errtheblog.com/s/regex) have slightly different cheatsheets\n",
    "  * `gem install cheat` for command-line tool or see <http://cheat.errtheblog.com>\n",
    "* [http://www.regular-expressions.info/tutorialcnt.html](http://www.regular-expressions.info/) has a good tutorial"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Ruby 2.4.1",
   "language": "ruby",
   "name": "ruby"
  },
  "language_info": {
   "file_extension": ".rb",
   "mimetype": "application/x-ruby",
   "name": "ruby",
   "version": "2.4.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}