Skip to content

Web and Browser Stuff

Jump to bottom
DeadlyCoconuts edited this page Jul 6, 2019 · 4 revisions
overview2

The backend of Wikipedia Web Mapper is powered by Flask and Jinja, and is handled mainly by the script main.py. The front-end is on the other hand mainly managed by the templates search.html and result.html, and style.css to provide basic styling. More details on the graph creation is provided under the Graph Creation section.

Table of Contents

The Main Page

On the back-end

Nothing too complicated here; the function render_template() in index() simply gets called to display the search.html template. The 3 search options are also passed into the function to be displayed on the template as a list of strings:

@app.route("/")
def index():
    return render_template("search.html", 
    options = ["Search for links from the 1st paragraph", "Search for links from the introduction", "Search for links from the entire page"]) # displays initial search bar

On the front-end

The HTML template for the home page is pretty simple too; there are 2 input bars for the URL source and the search depth, 1 drop-down list for the search options and a submit button. Jinja is used to display the search options passed to the render_template() as options in the drop-down list:

<form class="form-inline" method="POST" action="{{ url_for('result')}}">
    <!--Search Form-->
    <div class="input-bar">
        <input type="text" name="search_body" placeholder="Enter URL here..">
    </div> 
    <div class="input-bar">
        <input type="text" name="max_level" placeholder="Enter MAX search depth here..">
    </div>
    <!--Dropdown list-->
    <div class="select-bar">
        <select name="comp_select" class="selectpicker form-control">
            {% for option in options %}
            <option value="{{ option }}">
                {{ option }}
            </option>
            {% endfor %}
        </select>
    </div>
    <!--Button-->
    <button type="submit" class="btn">Create my graph!</button>
</form>

The Results Page

On the back-end

Once the form above has been submitted, the page /result gets called, triggering the function result() in main.py.

The first thing that occurs involves getting all the user input that has been entered in the form of Python variables. Since there are 3 user input fields, there are 3 corresponding variables that we'd expect to create - source_url, max_level and search_type. The last variable uastring created doesn't come from a user input - it comes from the type of browser the user was accessing the application from:

source_url = request.form.get('search_body') # gets search URL from fill-in form
max_level = request.form.get('max_level') # gets max search depth
search_type = request.form.get('comp_select') # gets search mode
uastring = request.headers.get('user_agent') # gets browser type

A simple if-elif block is then used to convert the search type to an integer to avoid passing clunky strings across functions:

# interprets search type picked by user
if search_type == "Search for links from the 1st paragraph":
    search_mode = 1
elif search_type == "Search for links from the introduction":
    search_mode = 2
elif search_type == "Search for links from the entire page":
    search_mode = 3
else:
    search_mode = 0 # or throw some error

The uastring variable is also analysed to determine if the user is indeed accessing the application from a mobile browser:

# checks browser type of user
isMobileBrowser = False
if "Mobile" in uastring:
    isMobileBrowser = True

With all the user input cleaned up, it's time to initiate the web scraping process to retrieve all the nodes and edges needed to create the graph. To do this, the function generate_lists() from process_link.py is called:

# data processing
if source_url: # to edit throw error/exception
    nodeList, linkList = generate_lists(source_url, max_level, isMobileBrowser, search_mode)

Last but not least, both the list of nodes and edges nodeList and linkList get passed into the HTML template result.html:

return render_template("result.html", nodeList = nodeList, linkList = linkList)

On the front-end

What then happens on result.html is really minor because most of the page is really just defined in the Javascript file script.js. However, there are still some things that ought to be noted:

Since D3.js will be used to create our graph, the client needs to have a local copy of the library for the script.js to function. This will be taken care of by loading the library directly from the internet (this means that this application will fail to work even on your development server if you have no access to the internet):

<script src="https://d3js.org/d3.v4.min.js"></script>

Just before loading the script.js to create our graph, we'll need to retrieve the nodeList and linkList objects that we have created previously in main.py. This is easy with the use of Jinja expressions in result.html. All that's left to do after that is to load script.js:

<script>
    var baseNodes = {{ nodeList | tojson }};
    var baseLinks = {{ linkList | tojson }};
</script>
<script src="{{ url_for('static', filename='js/script.js') }}"></script>

Refer to the Graph Creation section for more details on how the graph is actually created.

Some Basic Styling

Besides the usual page elements, there is just a small little section defining the page elements in search.html, varying their width to better fit the dimensions of mobile browsers:

.center-on-page {
  position: absolute;
  top: 50%;
  left: 50%;
  transform: translate(-50%, -50%);
}
/* Uses alternate settings if screen size is below 600px */
@media only screen and (max-width: 600px) { 
  .center-on-page {
    width: 80%;
  }
}
Clone this wiki locally