{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# TP1 - Finding Keys using Functional Dependencies \n", "--------------------------\n", "\n", "In this lab, we are going to learn \n", "\n", "- how to use Jupyter\n", "- how to use SQLite\n", "- how to discover keys in relations\n", "\n", "## How to use Jupyter\n", "\n", "To execute a cell, click on it, and then use SHIFT+ENTER (this will execute the contents of the cell, and move down to the next one!)\n", "\n", "To edit a cell, click on it. If the cell uses markdown code, then use ENTER to edit it.\n", "\n", "See the Help menu for other useful keyboard commands. You can always use the menu bar instead as well.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Hello world!\n" ] } ], "source": [ "print(\"Hello world!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another example:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "2\n", "3\n", "4\n", "5\n", "6\n", "7\n", "8\n", "9\n" ] } ], "source": [ "for i in range(1,10):\n", " print(i)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Exercise 1\n", "\n", "Print numbers 1 to 9 in reverse order" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "9\n", "8\n", "7\n", "6\n", "5\n", "4\n", "3\n", "2\n", "1\n" ] } ], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How to use SQLite\n", "\n", "To work with SQL easily in a notebook, we'll load the ipython-sql extension as follows:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Connected: None@None'" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%load_ext sql\n", "%sql sqlite://" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We are going to create a table:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Done.\n", "Done.\n", "1 rows affected.\n", "1 rows affected.\n", "1 rows affected.\n" ] }, { "data": { "text/plain": [ "[]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%sql DROP TABLE IF EXISTS T;\n", "CREATE TABLE T(course VARCHAR, classroom INT, time INT);\n", "INSERT INTO T VALUES ('CS 364', 132, 900);\n", "INSERT INTO T VALUES ('CS 245', 140, 1000);\n", "INSERT INTO T VALUES ('EE 101', 210, 900);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's run our first SQL query:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Done.\n" ] }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
courseclassroomtime
CS 364132900
CS 2451401000
EE 101210900
" ], "text/plain": [ "[('CS 364', 132, 900), ('CS 245', 140, 1000), ('EE 101', 210, 900)]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%sql SELECT * FROM T;" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Exercise 2\n", "\n", "List the name of the courses with time less than 950" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Done.\n" ] }, { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
course
CS 364
EE 101
" ], "text/plain": [ "[('CS 364',), ('EE 101',)]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How to discover keys in relations\n", "\n", "Now, we are going to work with functional dependencies, keys and closures. Our final goal is going to build a method to find keys in a relation.\n", "\n", "### Functional Dependencies\n", "\n", "Recall that given a set of attributes $\\{A_1, \\dots, A_n\\}$ and a set of FDs $\\Gamma$\n", "\n", "The closure, denoted $\\{A_1, \\dots, A_n\\}^+$, is defined to be the largest set of attributes B s.t. $$A_1,\\dots,A_n \\rightarrow B \\text{ using } \\Gamma.$$\n", "\n", "We're going to use some functions to compute the closure of a set of attributes and other such operations (_from CS145 Stanford_)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# Source code\n", "\n", "def to_set(x):\n", " \"\"\"Convert input int, string, list, tuple, set -> set\"\"\"\n", " if type(x) == set:\n", " return x\n", " elif type(x) in [list, set]:\n", " return set(x)\n", " elif type(x) in [str, int]:\n", " return set([x])\n", " else:\n", " raise Exception(\"Unrecognized type.\")\n", "\n", "def fd_to_str(lr_tuple):\n", " lhs = lr_tuple[0]\n", " rhs = lr_tuple[1]\n", " return \",\".join(to_set(lhs)) + \" -> \" + \",\".join(to_set(rhs))\n", "\n", "def fds_to_str(fds): return \"\\n\\t\".join(map(fd_to_str, fds))\n", "\n", "def set_to_str(x): return \"{\" + \",\".join(x) + \"}\"\n", "\n", "def fd_applies_to(fd, x): \n", " lhs, rhs = map(to_set, fd)\n", " return lhs.issubset(x)\n", "\n", "def print_setup(A, fds):\n", " print(\"Attributes = \" + set_to_str(A))\n", " print(\"FDs = \\t\" + fds_to_str(fds))\n", "\n", "def print_fds(fds):\n", " print(\"FDs = \\t\" + fds_to_str(fds)) \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's look at a concrete example. For example, the code for\n", "\n", "attributes = { name, category, color, department, price}\n", "\n", "and functional dependencies:\n", "\n", "name $\\rightarrow$ color\n", "\n", "category $\\rightarrow$ department\n", "\n", "color, category $\\rightarrow$ price\n", "\n", "is the following:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Attributes = {department,color,category,name,price}\n", "FDs = \tname -> color\n", "\tcategory -> department\n", "\tcolor,category -> price\n" ] } ], "source": [ "attributes = set([\"name\", \"category\", \"color\", \"department\", \"price\"]) # These are the attribute set.\n", "fds = [(set([\"name\"]),\"color\"),\n", " (set([\"category\"]), \"department\"),\n", " (set([\"color\", \"category\"]), \"price\")]\n", "\n", "print_setup(attributes, fds)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Closure of a set of Attributes\n", "\n", "Let's implement the algorithm for obtaining the closure of a set of attributes:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "def compute_closure(x, fds, verbose=False):\n", " bChanged = True # We will repeat until there are no changes.\n", " x_ret = x.copy() # Make a copy of the input to hold x^{+}\n", " while bChanged:\n", " bChanged = False # Must change on each iteration\n", " for fd in fds: # loop through all the FDs.\n", " (lhs, rhs) = map(to_set, fd) # recall: lhs -> rhs\n", " if fd_applies_to(fd, x_ret) and not rhs.issubset(x_ret):\n", " x_ret = x_ret.union(rhs)\n", " if verbose:\n", " print(\"Using FD \" + fd_to_str(fd))\n", " print(\"\\t Updated x to \" + set_to_str(x_ret))\n", " bChanged = True\n", " return x_ret" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As an example, let's compute the closure for the attribute \"name\":" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Using FD name -> color\n", "\t Updated x to {color,name}\n" ] }, { "data": { "text/plain": [ "{'color', 'name'}" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "A = set([\"name\"])\n", "compute_closure(A,fds, True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Exercise 3\n", "\n", "Is the attribute \"name\" a superkey for this relation? Why?" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "Answer: Write your answer here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Keys and Superkeys\n", "\n", "Next, we'll add some new functions now for finding superkeys and keys. Recall:\n", "* A _superkey_ for a relation $R(B_1,\\dots,B_m)$ is a set of attributes $\\{A_1,\\dots,A_n\\}$ s.t.\n", "$$ \\{A_1,\\dots,A_n\\} \\rightarrow B_{j} \\text{ for all } j=1,\\dots m$$\n", "* A _key_ is a minimal (setwise) _superkey_\n", "\n", "The algorithm to determine if a set of attributes $A$ is a superkey for $X$ is actually very simple (check if $A^+=X$):" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "def is_superkey_for(A, X, fds, verbose=False): \n", " return X.issubset(compute_closure(A, fds, verbose=verbose))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Is \"name\" a superkey of the relation?" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "is_superkey_for(A, attributes, fds)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, to check if $A$ is a key for $X$, we just confirm that:\n", "* (a) it is a superkey\n", "* (b) there are no smaller superkeys (_Note that we only need to check for superkeys of one size smaller_)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "import itertools\n", "def is_key_for(A, X, fds, verbose=False):\n", " subsets = set(itertools.combinations(A, len(A)-1))\n", " return is_superkey_for(A, X, fds) and \\\n", " all([not is_superkey_for(set(SA), X, fds) for SA in subsets])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's look at another example:\n", "\n", "attributes = { cru, type, client, remise}\n", "\n", "and functional dependencies:\n", "\n", "cru $\\rightarrow$ type\n", "\n", "type, client $\\rightarrow$ remise\n", "\n", "#### Exercise 4\n", "\n", "Is \"cru\" and \"client\" a key of the relation? Why?" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because it's a superkey and it is minimal. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Closure of a set of functional dependencies\n", "\n", "The algorithm to find the closure of a set of functional dependencies is the following:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "import itertools\n", "def findsubsets(S,m):\n", " return set(itertools.combinations(S, m))\n", "def closure(X, fds, verbose=False):\n", " c = []\n", " for size in range(1, len(X)):\n", " subsets = findsubsets(X, size) \n", " for SA in subsets: # loop through all the subsets.\n", " cl = compute_closure(set(SA), fds, verbose)\n", " if len(cl.difference(SA)) > 0: \n", " c.extend([(set(SA), cl.difference(SA))])\n", " return c" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see some examples of how to use it:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "FDs = \tB -> D\n", "\tC,B -> D\n", "\tA,B -> D,C\n", "\tD,A -> C,B\n", "\tD,A,B -> C\n", "\tA,D,C -> B\n", "\tA,C,B -> D\n" ] } ], "source": [ "attributes1 = set(['A', 'B', 'C', 'D'])\n", "fds1 = [(set(['A', 'B']), 'C'),\n", " (set(['A', 'D']), 'B'),\n", " (set(['B']), 'D')]\n", "print_fds(closure(attributes1, fds1))\n" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "FDs = \tCRU -> TYPE\n", "\tREMISE,CRU -> TYPE\n", "\tCRU,CLIENT -> REMISE,TYPE\n", "\tTYPE,CLIENT -> REMISE\n", "\tTYPE,CRU,CLIENT -> REMISE\n", "\tREMISE,CRU,CLIENT -> TYPE\n" ] } ], "source": [ "attributes2 = set (['CRU', 'TYPE', 'CLIENT', 'REMISE'])\n", "fds2 = [(set(['CRU']), 'TYPE'),\n", " (set(['TYPE', 'CLIENT']), 'REMISE')]\n", "print_fds(closure(attributes2, fds2))" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "FDs = \tTYPE -> PUISSANCE,MARQUE\n", "\tN VEH -> PUISSANCE,COULEUR,TYPE,MARQUE\n", "\tN VEH,PUISSANCE -> COULEUR,TYPE,MARQUE\n", "\tTYPE,MARQUE -> PUISSANCE\n", "\tPUISSANCE,TYPE -> MARQUE\n", "\tCOULEUR,TYPE -> PUISSANCE,MARQUE\n", "\tN VEH,TYPE -> COULEUR,PUISSANCE,MARQUE\n", "\tN VEH,MARQUE -> COULEUR,PUISSANCE,TYPE\n", "\tN VEH,COULEUR -> PUISSANCE,TYPE,MARQUE\n", "\tN VEH,COULEUR,MARQUE -> PUISSANCE,TYPE\n", "\tN VEH,PUISSANCE,MARQUE -> COULEUR,TYPE\n", "\tCOULEUR,TYPE,MARQUE -> PUISSANCE\n", "\tN VEH,TYPE,MARQUE -> COULEUR,PUISSANCE\n", "\tPUISSANCE,COULEUR,TYPE -> MARQUE\n", "\tN VEH,COULEUR,TYPE -> PUISSANCE,MARQUE\n", "\tN VEH,PUISSANCE,TYPE -> COULEUR,MARQUE\n", "\tN VEH,PUISSANCE,COULEUR -> TYPE,MARQUE\n", "\tN VEH,PUISSANCE,TYPE,MARQUE -> COULEUR\n", "\tN VEH,COULEUR,TYPE,MARQUE -> PUISSANCE\n", "\tN VEH,PUISSANCE,COULEUR,MARQUE -> TYPE\n", "\tN VEH,PUISSANCE,COULEUR,TYPE -> MARQUE\n" ] } ], "source": [ "attributes3 = set( ['N VEH', 'TYPE', 'COULEUR', 'MARQUE', 'PUISSANCE'])\n", "fds3 = [(set(['N VEH']), 'TYPE'), \n", " (set(['N VEH']), 'COULEUR'),\n", " (set(['TYPE']), 'MARQUE'),\n", " (set(['TYPE']), 'PUISSANCE')]\n", "print_fds(closure(attributes3,fds3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's write a method to find superkeys of the relations:\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "def superkeys(X, fds, verbose=False):\n", " c = []\n", " for size in range(1, len(X)):\n", " subsets = findsubsets(X, size)\n", " for SA in subsets:\n", " cl = compute_closure(set(SA), fds, verbose)\n", " if cl == X and len(cl.difference(SA)) > 0: ## cl = X\n", " c.extend([SA])\n", " return c" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('A', 'B'), ('D', 'A'), ('D', 'A', 'B'), ('C', 'D', 'A'), ('C', 'A', 'B')]" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "superkeys(attributes1, fds1)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see some examples:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('CLIENT', 'CRU'), ('CLIENT', 'TYPE', 'CRU'), ('CLIENT', 'REMISE', 'CRU')]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "superkeys(attributes2, fds2)\n" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('N VEH',),\n", " ('N VEH', 'PUISSANCE'),\n", " ('N VEH', 'TYPE'),\n", " ('N VEH', 'MARQUE'),\n", " ('N VEH', 'COULEUR'),\n", " ('N VEH', 'COULEUR', 'MARQUE'),\n", " ('N VEH', 'PUISSANCE', 'MARQUE'),\n", " ('N VEH', 'TYPE', 'MARQUE'),\n", " ('N VEH', 'COULEUR', 'TYPE'),\n", " ('N VEH', 'PUISSANCE', 'TYPE'),\n", " ('N VEH', 'PUISSANCE', 'COULEUR'),\n", " ('N VEH', 'PUISSANCE', 'TYPE', 'MARQUE'),\n", " ('N VEH', 'COULEUR', 'TYPE', 'MARQUE'),\n", " ('N VEH', 'PUISSANCE', 'COULEUR', 'MARQUE'),\n", " ('N VEH', 'PUISSANCE', 'COULEUR', 'TYPE')]" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "superkeys(attributes3, fds3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Exercise 5\n", "\n", "Implement a `keys` method to find keys of a relation.\n", "\n", "**Note**: If there exist multiple keys of a relation, the `keys` method should return at least one of them." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Test it " ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'A', 'B'}" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "keys(attributes1, fds1)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'CLIENT', 'CRU'}" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "keys(attributes2, fds2)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'N VEH'}" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "keys(attributes3, fds3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [conda root]", "language": "python", "name": "conda-root-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.4" } }, "nbformat": 4, "nbformat_minor": 2 }