asr_lab5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 5  - WFST operations\n",
    "\n",
    "So far we've used WFSTs mainly as a usual structure for encoding and traversing HMMs.  In this lab we'll move away from HMM acoustic modelling and look at how WFST operations can be used to avoid the need for specialised algorithms in speech and language processing.  It is intended to give you insight into how these operations are used to construct HMMs encapsulating langauge model, pronunciation and acoustic modelling assumptions &ndash; the so-called \"HCLG\" WFST.\n",
    "\n",
    "This lab will focus on the lexicon transducer, $L$, and grammar transducer, $G$.\n",
    "\n",
    "We'll use some of the following operations, defined by Openfst:\n",
    "* `fst.determinize(f)` creates determinized version of `f`\n",
    "* `fst.compose(f1,f2)` composes FSTs `f1` and `f2`\n",
    "* `fst.shortestpath(f)` returns the shortest path (in terms of weight) through `f` from the start to a final state\n",
    "* `f.minimize()` creates minimized version of `f`\n",
    "* `f.project(project_output=False)` for every arc in `f`, copies the input label to the output label (or vice versa, if `project_output=True`).\n",
    "* `f.rmepsilon()` removes epsilon transitions &ndash; those arcs where both input and output labels are empty.\n",
    "\n",
    "For efficiency, the compostion of `f1` and `f2` requires either the output arcs of `f1` or input arcs of `f2` to be sorted prior to `compose()` being called.  You can do this by calling `f1.arcsort(sort_type='olabel')` or `f2.arcsort(sort_type='ilabel')`.\n",
    "\n",
    "Combining projection and removal of epsilon transitions will give you better visual clarity when looking at your FSTS.\n",
    "\n",
    "The functions above assume that `openfst_python` has been imported as `fst`.  Note that the first three functions above return a new WFST; the others modify the WFST *in place*, meaning that the original WFST is modified directly.\n",
    "\n",
    "For convenience, we've provided a python module `helper_functions` that provides the `parse_lexicon()` and `generate_symbol_tables()` from the [Lab 1 solutions](https://github.com/Ore-an/asr_lab1/blob/master/asr_lab1_solutions.ipynb).  And here is a function to generate an $L$ transducer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "partner = 's1608733'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from subprocess import check_call\n",
    "from IPython.display import Image\n",
    "import openfst_python as fst\n",
    "from helper_functions import parse_lexicon, generate_symbol_tables\n",
    "\n",
    "lex = parse_lexicon('lexicon.txt')\n",
    "word_table, phone_table, state_table = generate_symbol_tables(lex)  # we won't use state_table in this lab\n",
    "\n",
    "def generate_L_wfst(lex):\n",
    "    \"\"\" Express the lexicon in WFST form\n",
    "    \n",
    "    Args:\n",
    "        lexicon (dict): lexicon to use, created from the parse_lexicon() function\n",
    "    \n",
    "    Returns:\n",
    "        the constructed lexicon WFST\n",
    "    \n",
    "    \"\"\"\n",
    "    L = fst.Fst()\n",
    "    \n",
    "    # create a single start state\n",
    "    start_state = L.add_state()\n",
    "    L.set_start(start_state)\n",
    "    \n",
    "    for (word, pron) in lex.items():\n",
    "        \n",
    "        current_state = start_state\n",
    "        for (i,phone) in enumerate(pron):\n",
    "            next_state = L.add_state()\n",
    "            \n",
    "            if i == len(pron)-1:\n",
    "                # add word output symbol on the final arc\n",
    "                L.add_arc(current_state, fst.Arc(phone_table.find(phone), \\\n",
    "                                                 word_table.find(word), None, next_state))\n",
    "            else:\n",
    "                L.add_arc(current_state, fst.Arc(phone_table.find(phone),0, None, next_state))\n",
    "            \n",
    "            current_state = next_state\n",
    "                          \n",
    "        L.set_final(current_state)\n",
    "        \n",
    "        # add epsilon transition to start state to recognize multiple words (exercise 3)\n",
    "        L.add_arc(current_state, fst.Arc(0, 0, None, 0))\n",
    "        \n",
    "    L.set_input_symbols(phone_table)\n",
    "    L.set_output_symbols(word_table)                      \n",
    "    \n",
    "    return L\n",
    "\n",
    "L = generate_L_wfst(lex)\n",
    "L.arcsort()\n",
    "L.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the exercises, here are two functions to generate linear WFSTs for an arbitary sequence of phones or words.  (Yes, they are really just variants of the same function!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_linear_phone_wfst(phone_list):\n",
    "    \n",
    "    P = fst.Fst()\n",
    "    \n",
    "    current_state = P.add_state()\n",
    "    P.set_start(current_state)\n",
    "    \n",
    "    for p in phone_list:\n",
    "        \n",
    "        next_state = P.add_state()\n",
    "        P.add_arc(current_state, fst.Arc(phone_table.find(p), phone_table.find(p), None, next_state))\n",
    "        current_state = next_state\n",
    "        \n",
    "    P.set_final(current_state)\n",
    "    P.set_input_symbols(phone_table)\n",
    "    P.set_output_symbols(phone_table)\n",
    "    return P\n",
    "    \n",
    "def generate_linear_word_wfst(word_list):\n",
    "    \n",
    "    W = fst.Fst()\n",
    "    \n",
    "    current_state = W.add_state()\n",
    "    W.set_start(current_state)\n",
    "    \n",
    "    for w in word_list:\n",
    "        \n",
    "        next_state = W.add_state()\n",
    "        W.add_arc(current_state, fst.Arc(word_table.find(w), word_table.find(w), None, next_state))\n",
    "        current_state = next_state\n",
    "        \n",
    "    W.set_final(current_state)\n",
    "    W.set_input_symbols(word_table)\n",
    "    W.set_output_symbols(word_table)\n",
    "    return W"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "1. Suppose you are given a sequence of phones, in the form `['p','ih','k','t']`, and the $L$ transducer created above.  Write a function that returns the matching word from the lexicon for any given phone sequence, or else `None` if no matching word is found.   Write two functions:\n",
    "  1. That works for $L$ as provided by the code above\n",
    "  2. That works only on a determinized version of $L$ &ndash; and test it on the output of `fst.determinize(L)`\n",
    "  \n",
    " This should enable you to see why determinization is a very useful WFST operation!\n",
    "  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "def transduce_sequence_nondet(f, in_seq):\n",
    "    \"\"\"Return transduced sequence given input sequence and non determinized FST.\n",
    "        Note that our skeleton code uses a queue to traverse the FST.\n",
    "        You can change this if you prefer, as long as you correctly output the list of trasduced words.\n",
    "    \n",
    "        Args:\n",
    "            f (fst.Fst()): a non determinized FST\n",
    "            in_seq (list[str]): the sequence of strings to transduce\n",
    "            \n",
    "        Returns:\n",
    "            out_seq (list[str]): the sequence of transduced symbols\n",
    "            \"\"\"\n",
    "    \n",
    "    seq_len = len(in_seq)\n",
    "    in_seq.append('<EOS>') # adding a padding symbol at the end for possible final eps traversal\n",
    "    queue = [(f.start(), 0, [])]  # the tuple is (state, index in input sequence, output)\n",
    "    outputs = []\n",
    "    \n",
    "    while queue:\n",
    "        \n",
    "        curr_state, i, output = queue.pop(0) # pop first element in list\n",
    "        \n",
    "        if i <= seq_len:  # <= because we could traverse epsilons even when the input sequence ended\n",
    "            \n",
    "            # iterate over all arcs from current state\n",
    "            for arc in f.arcs(curr_state):\n",
    "                \n",
    "                if phone_table.find(arc.ilabel) == in_seq[i]:\n",
    "                    \n",
    "                    queue.append((arc.nextstate, i + 1, arc.olabel))\n",
    "                    outputs.append(arc.olabel)\n",
    "        \n",
    "        if i == seq_len:\n",
    "            final_weight = float(f.final(curr_state))\n",
    "            if final_weight != math.inf: # if this is a final state\n",
    "                # find the labels in the table, remove epsilons\n",
    "                out_seq = [f.output_symbols().find(w) for w in outputs if w != 0]\n",
    "                return out_seq\n",
    "            \n",
    "    # return exits the function, so this is printed only when the stack is empty and we didn't find a path \n",
    "    print(\"Can't transduce the sequence with provided FST\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq = ['p','ih','k','t']\n",
    "print(transduce_sequence_nondet(L, seq))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ldet = fst.determinize(L)\n",
    "Ldet.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def transduce_sequence_det(f, seq):\n",
    "    \"\"\"Return transduced sequence given input sequence and determinized FST\n",
    "    \n",
    "        Args:\n",
    "            f (fst.Fst()): a determinized FST\n",
    "            in_seq (list[str]): the sequence of strings to transduce\n",
    "            \n",
    "        Returns:\n",
    "            out_seq (list[str]): the sequence of transduced symbols\n",
    "            \"\"\"\n",
    "    \n",
    "    seq_len = len(seq)\n",
    "    curr_state = f.start()\n",
    "    output = []\n",
    "    \n",
    "    for i in range(seq_len):\n",
    "        \n",
    "        # iterate over all arcs from current state\n",
    "        for arc in f.arcs(curr_state):\n",
    "            \n",
    "            if phone_table.find(arc.ilabel) == seq[i]:\n",
    "                \n",
    "                curr_state = arc.nextstate\n",
    "                output.append(arc.olabel)\n",
    "        \n",
    "    final_weight = float(f.final(curr_state))\n",
    "    if final_weight != math.inf: # if this is a final state\n",
    "        # find the labels in the table, remove epsilons\n",
    "        out_seq = [f.output_symbols().find(w) for w in output if w != 0]\n",
    "        return out_seq\n",
    "    else:  \n",
    "        print(\"Can't transduce the sequence with provided FST\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq = ['p','ih','k','t']\n",
    "print(transduce_sequence_det(Ldet, seq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. WFST composition allows you to achieve the same result much more easily.  Create a linear WFST, $P$, corresponding to a string of phones, and compute $P \\circ L$.  Then use the projection and epsilon removal operations to display just the matching word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq = ['p', 'ih', 'k', 't']\n",
    "P = generate_linear_phone_wfst(seq)\n",
    "P"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sort output arcs of f1 or input arcs of f2\n",
    "# f1.arcsort(sort_type='olabel') or f2.arcsort(sort_type='ilabel')\n",
    "P = P.arcsort(sort_type='olabel')\n",
    "P_L = fst.compose(P, L)\n",
    "P_L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for every arc in f, copies the input label to the output label (or vice versa, if project_output=True)\n",
    "P_L.project(project_output=True)\n",
    "# removes epsilon transitions – those arcs where both input and output labels are empty\n",
    "P_L.rmepsilon()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Modify your lexicon WFST slightly to allow a list of phones to be \"decoded\" to a sequence of multiple words from the lexicon. Then, compose the modified lexicon with a sequence of phones of multiple concatenated words.  Try it with `['p','eh','k','ah','v','p','iy','t','er']`.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_word_phones = ['p','eh','k','ah','v','p','iy','t','er']\n",
    "P = generate_linear_phone_wfst(seq_word_phones)\n",
    "\n",
    "P.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "P = P.arcsort(sort_type='olabel')\n",
    "P_L = fst.compose(P, L)\n",
    "\n",
    "P_L.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "P_L.project(project_output=True)\n",
    "P_L.rmepsilon()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Now solve the reverse problem: create a word-sequence WFST, $W$, and use composition to expand it into a sequence of phones. You'll need to sort by output labels this time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_words = ['peck', 'of', 'peter']\n",
    "W = generate_linear_word_wfst(seq_words)\n",
    "W"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = W.arcsort(sort_type='olabel')\n",
    "W_L = fst.compose(L, W)\n",
    "\n",
    "W_L.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W_L.project(project_output=False)\n",
    "W_L.rmepsilon()\n",
    "\n",
    "W_L.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Another advantage of WFST composition to solve these kind of problems are that it is easy to encode uncertainty in the input (a bit like in real ASR).  For example, consider this WFST, in which the multiple arcs denote alternative phone transcriptions from the acoustic model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_alt_phone_wfst(phone_alternatives):\n",
    "    \n",
    "    P = fst.Fst()\n",
    "    \n",
    "    current_state = P.add_state()\n",
    "    P.set_start(current_state)\n",
    "    \n",
    "    for alt in phone_alternatives:\n",
    "        \n",
    "        next_state = P.add_state()\n",
    "        for p in alt:\n",
    "            if p=='*':\n",
    "                P.set_final(current_state)\n",
    "            else:\n",
    "                P.add_arc(current_state, fst.Arc(phone_table.find(p), phone_table.find(p), None, next_state))\n",
    "        current_state = next_state\n",
    "        \n",
    "    P.set_final(current_state)\n",
    "    P.set_input_symbols(phone_table)\n",
    "    P.set_output_symbols(phone_table)\n",
    "    return P    \n",
    "    \n",
    "altP = create_alt_phone_wfst([['p'],['ay'],['p'],['er'],['p'],['eh','ih'],['k'],['t','<eps>'],['ah','<eps>'],['l','v','*'],['d','*']])\n",
    "altP.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "  Again, perform composition with your $L$ from Question 3, and observe the result.  (Notice particularly what happens to the `<eps>` transitions during composition.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "altP = fst.compose(altP, L)\n",
    "altP.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "altP.project(project_output=True)\n",
    "altP.rmepsilon()\n",
    "\n",
    "altP.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. We could have added weights to the arcs of the WFST above to describe the probability of the phone alternatives given by the acoustic model &ndash; this would have enabled you to find the most likely sequence of words.  Without this information, let's instead use a $G$ WFST to find the most likely sequence.  Let's assume that a word sequence taken from the passage \"peter piper picked a peck of pickled peppers\" is most likely.  Design a $G$ WFST that accepts any sequence of words from the lexicon, but adds a cost of 1.0 to any word transition not in the passage.  Given $G$, use composition to recover the most likely word sequence from the uncertain $P$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_G_wfst(wseq):\n",
    "    \"\"\" Generate a grammar WFST that accepts any sequence of words for words in a sentence.\n",
    "        The bigrams not present in the sentence have a cost of 1, while those present have a cost of 0. \n",
    "        Args:\n",
    "            wseq (str): the sentence to use\n",
    "        Returns:\n",
    "            W (fst.Fst()): the grammar WFST \"\"\"\n",
    "    \n",
    "    G = fst.Fst()\n",
    "    start_state = G.add_state()\n",
    "    G.set_start(start_state)\n",
    "    \n",
    "    # need a secondary start state to account for first token cost\n",
    "    restart_state = G.add_state()\n",
    "    \n",
    "    # special case for first token\n",
    "    w = word_table.find(wseq.split()[0])\n",
    "    first_state = G.add_state()\n",
    "    G.add_arc(start_state, fst.Arc(w, w, 0, first_state))\n",
    "    G.add_arc(first_state, fst.Arc(0, 0, 0, restart_state))\n",
    "    G.add_arc(first_state, fst.Arc(w, w, 1, restart_state))\n",
    "    G.add_arc(restart_state, fst.Arc(w, w, 1, first_state))\n",
    "    \n",
    "    prev_state = first_state\n",
    "    \n",
    "    # skip first token, because it has already been handled\n",
    "    for i, w in enumerate(wseq.split()[1:]):\n",
    "        \n",
    "        current_state = G.add_state()\n",
    "\n",
    "        w = word_table.find(w)\n",
    "        \n",
    "        # add arc from prev state\n",
    "        G.add_arc(prev_state, fst.Arc(w, w, 0, current_state))\n",
    "    \n",
    "        # add arc from start state\n",
    "        G.add_arc(start_state, fst.Arc(w, w, 1, current_state))\n",
    "        \n",
    "        # add arc from first state\n",
    "        if i >= 1:\n",
    "            G.add_arc(first_state, fst.Arc(w, w, 1, current_state))\n",
    "        \n",
    "        # add arc from restart state\n",
    "        G.add_arc(restart_state, fst.Arc(w, w, 1, current_state))\n",
    "        \n",
    "        # add epsilon transition\n",
    "        G.add_arc(current_state, fst.Arc(0, 0, 0, restart_state))\n",
    "\n",
    "        prev_state = current_state\n",
    "    \n",
    "    G.set_final(start_state)\n",
    "    G.set_final(restart_state)\n",
    "        \n",
    "    G.set_input_symbols(word_table)\n",
    "    G.set_output_symbols(word_table)  \n",
    " \n",
    "    return G\n",
    "\n",
    "string = \"peter piper picked a peck of pickled peppers\"\n",
    "G = generate_G_wfst(string)\n",
    "\n",
    "G.draw('tmp.dot', portrait=True)\n",
    "check_call(['dot','-Tpng','-Gdpi=200','tmp.dot','-o','tmp.png'])\n",
    "Image(filename='tmp.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now take the composition of the alternative pronunciation sequence and L you did in the exercise before and compose it with your new grammar. Check the output of the shortest path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "G.arcsort(sort_type='olabel')\n",
    "comp = fst.compose(altP, G)\n",
    "comp.rmepsilon()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fst.shortestpath(comp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## If you have more time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use WFST composition to implement a \"predictive text\"-style algorithm, that, given a partial phone sequence such as `['p']` or `['p','ih']`, returns a WFST giving all matching words.  You'll need to make some special modifications to $P$ or $L$, or both. On a determinized $L$ transducer this is a highly efficient way of solving this problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# There are very many ways this problem can be solved. Our skeleton code add extra arcs with \n",
    "# special <rho> symbol. This symbol will represent unterminated sequences, and will be transduced\n",
    "#  to words to be output at every intermediate state.\n",
    "# The same <rho> symbol is added the end of the partial pronunciation.\n",
    "\n",
    "def generate_predictive_L_wfst(lex):\n",
    "    \"\"\" express the lexicon in WFST form s.t. composition with partial sequence gives matching words\n",
    "    \n",
    "    Args:\n",
    "        lexicon (dict): lexicon to use, created from the parse_lexicon() function\n",
    "    \n",
    "    Returns:\n",
    "        the constructed WFST\n",
    "    \n",
    "    \"\"\"\n",
    "    Lpred = fst.Fst()\n",
    "    rho = phone_table.add_symbol('<rho>')\n",
    "    # create a single start state\n",
    "    start_state = Lpred.add_state()\n",
    "    Lpred.set_start(start_state)\n",
    "    \n",
    "    for (word, pron) in lex.items():\n",
    "        \n",
    "        # Your code here\n",
    "        \n",
    "    Lpred.set_input_symbols(phone_table)\n",
    "    Lpred.set_output_symbols(word_table)                      \n",
    "    \n",
    "    return Lpred\n",
    "\n",
    "Lpred = generate_predictive_L_wfst(lex)\n",
    "Lpred.arcsort()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq = ['p']\n",
    "seq += ['<rho>']\n",
    "Ppred = generate_linear_phone_wfst(seq)\n",
    "Ppred.arcsort(sort_type='ilabel')\n",
    "comp = fst.compose(Ppred, Lpred)\n",
    "comp.project(True).rmepsilon()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}