{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c565b776",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Amdahl's Law Applied\n",
    "==================\n",
    "\n",
    "\n",
    "* Ideally: code on 4 cores runs 4x faster than on 1.\n",
    "* That is rare. There is a name for it: \"Embarassingly Parallel.\"\n",
    "* It should really be called \"Perfectly Parallel.\"\n",
    "* Usually, application *Speedup* is limited by how it is designed or implemented.\n",
    "* When using large numbers or processes and processors, this hits *HARD.*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbac9108",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "\n",
    "Speedup\n",
    "-------------\n",
    "\n",
    "If you have a single processor to run code, then assuming perfect parallism (the so-called \"embarrasingly parallel\" case) then your code will run twice as fast on two processors, four times as fast on four, etc... This is the ideal case. Much real-life code has \"dependencies\" that limit speedup.\n",
    "\n",
    "Speedup of a task, when adding processors is limited by how much of the code can run in parallel. If you had an infinite number of processors to apply to a problem, then performance is limited by the p the parallelizability of your code.\n",
    "\n",
    "The graph below plots the speedup available against the degree to which the code is parallelizable (from 5% to 95% in steps of 5%.) For example, if the algorithm is only 50% parallelizable, then throwing an infinite number of processors at the problem will give you a 25% speedup ( 1.25 on the plot.) so it isn't worthwhile.\n",
    "\n",
    "Note that, towards the right edge of the plot, the speedup is nearly vertical. This means that the difference in real life performance near that limit, say between 98% and 99% parallelizable, is huge (as in closer to ten fold than 10%)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "588983a6",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7bf6e23a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": [
     "hide-input",
     "remove-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA2nUlEQVR4nO3dd5xcZb3H8c93e0/dhBSSUBIgIM3QWwCpKqCigiBNpVi4XLter2C7tmsFEbhKU4ogIIih11ACCUgghRAICUl2k9203c328rt/nGeSyTK7O7vJzGz5vV+vec3p5zdnZ+d3zvOc8zwyM5xzzrnOsjIdgHPOuf7JE4RzzrmEPEE455xLyBOEc865hDxBOOecS8gThHPOuYQ8QbjtJulpSZ/PdBw7iqTlkj6Uom1fJ+m/u5l/laS/7qB9TZK0WVL2jtheKsR/dyRdIOm5JNe7WdKPw/BRkpakMs6hyhPEACbpSEkvSKqRtEHS85IOynRcA4Gk4vDjOSud+zWzS83sRyGGmZJWpXBf75lZiZm192a98EPdHo5PraTXJH0kVXFuLzObbWZ7ZDqOwcgTxAAlqQx4ELgaGAlMAH4ANGcyrgHkTKJjdaKkcenYYX8+k0/gRTMrAYYDfwbukjSyNxuQlJOKwFz6eIIYuKYBmNkdZtZuZo1m9qiZvQ5bzgKfl3R1uMJ4U9LxsZUlDZP0Z0mVklZL+nH8D5ikiyQtlrRR0iOSJsfNOyFsr0bSNYDi5m1TRCJpiiSL/ViEIoWfSno5rH9/Vz88Yf8fiRvPkbRO0oGSCiT9VdJ6SZskzZU0thfH73zgOuB14JyuFpJUKOmWcBwWS/pm/Fm/pL3CZ9okaaGk0+Lm3Szpj5JmSaoHjo0VjUgqBh4Cxocz9c2SxodV8yTdKqkubHNG3DaXS/qGpNcl1Ye/4VhJD4XlH5c0ootjP1LSTZIqwuf5R08Hycw6gBuBQmBXSReG41AnaZmkS+JimylplaRvSVoD3CRphKQHJVWHfT4oaWLPfx6QtKekxxRdHS+R9KkultvmSizsf3WIcUnsex++m3eH702dpDckTZP0HUlVklZKOjGZ2IYKTxAD11tAe/jxOiX2o9DJIcAyYDRwJXBv3I/xLUAbsDtwAHAiECsLPgP4LvBxoByYDdwR5o0G7gG+F7b7DnBEL2M/D7gIGB9i+H0Xy90BnB03fhKwzsxeJfqBHwbsDIwCLgUak9m5pEnATOC28Dqvm8WvBKYAuwInAOfGbScX+CfwKDAG+Apwm6T44o7PAD8BSoEt5etmVg+cAlSEYqASM6sIs08D7iQ6e38AuKZTTJ8IsUwDPkqUaL5L9PfIAi7v4rP8BSgC9g7x/qabzx37jDlE34vNwFKgCvgIUAZcCPxG0oFxq+xEdEU7Gbg4xHNTGJ9E9Dfq/HkS7bcYeAy4PcR6NnCtpL17WG8P4MvAQWZWSvSdWR63yEeJjsMI4N/AIyHGCcAPget7im1IMTN/DdAXsBdwM7CK6If2AWBsmHcBUAEobvmXgc8CY4mKVwrj5p0NPBWGHwI+FzcvC2gg+ic/D5gTN09h/58P41cBf42bPwUwICeMPw38LG7+dKAFyE7w+XYH6oCiMH4b8P0wfBHwArBvH47b94DXwvB4oB04IG7+cuBDYXgZcFLcvM8Dq8LwUcAaICtu/h3AVWH4ZuDWTvu+GfhxGJ4Z21bc/KuAxzsdn8ZOsZ0TN34P8Me48a8A/+h87IFxQAcwIonjc0H4Pm0C1gFzYscjwbL/AP4j7vO0AAXdbHt/YGPc+NNx350LgOfC8KeB2Z3WvR64srvjGL4zVcCHgNwEx/axuPGPEiW+7DBeGo7X8HT/L/fXl19BDGBmttjMLjCzicA+RD92v41bZLWFb36wIiwzGcgFKkPRyCaif74xYbnJwO/i5m0gSgQTwvor42Kw+PEkxS+/IsQyOsHnextYDHxUUhHRmfXtYfZfiM7+7gxFJr8IZ/TJOI8o2WDRWfszRFckiWzzeTsNjwdWWlQME/95JnSxfLLWxA03AAXatjx/bdxwY4LxkgTb3BnYYGYbk4xhjpkNN7PRZnaomT0OEK5W54Rin03AqWz7t6s2s6bYiKQiSddLWiGpFngWGK6e62MmA4fEvoNhX+cQXaF0KXxnriBKBlWS7owruoP3H6t1trUSP3YFmuj4DUmeIAYJM3uT6Kxqn7jJEyQpbnwS0VXFSqIriNHhR2C4mZWZWezyfSVwSdy84WZWaGYvAJVEPzYAhO3vHLePeqJijJhE/9Dxy08CWonOVBOJFTOdDiwKPwCYWauZ/cDMpgOHExV7dFdUFIv3cGAq8B1Ja0JZ+SHA2UpcqVoJxJeZx8deAewsKf7/aBKwOm68u+aS09mU8kpgpKThfd2ApHyiK5b/JbpSHQ7MIq4Oivd/pq8BewCHmFkZcHRsc0nE+0yn72CJmV3WU5xmdruZHUmUZAz4eU/ruMQ8QQxQoQLva7EKP0k7E/2QzolbbAxwuaRcSZ8kKpKaZWaVROXmv5JUJilL0m6SjgnrXUf0A7p32PawsD7Av4C9JX08/KBezrZJ4DXgaEX34A8DvpMg/HMlTQ9XBT8E/m5d34p5J1H9yGVsvXpA0rGSPhDORGuJkkwyt3OeT1S2PZ2ouGN/oqRaRFQn0NldRMdihKQJROXbMS8RJcRvhmM8k6jY4s4k4oDobHZUOE4pFf7mDxGV448I8R7d03qd5AH5QDXQJukUor9Nd0qJzsw3hfqvK5Pc14PANEmfDbHmSjpI0l7drSRpD0nHhWTWFPbdq9t83VaeIAauOqIz35cU3SEzB1hAdMYW8xLR2fI6oorSM81sfZh3HtE//CJgI/B3onJqzOw+orOuO0OxwALCj6eZrQM+CfwMWB+2/3xsh2b2GPA3oruDXiH6R+/sL0RXO2uAArquVI39sL1IdJXwt7hZO4WYa4mKoZ4B/gpbHka7rvO2JBUAnwKuNrM1ca93Q0yJipl+SFTH8i7weNhnc4ithajY6xSiY3wtcF64mutRWO4OYFkoRhnf0zrb6bNEifRNonL6K3qzspnVEf2t7iL6znyGqN6rO78lugMqVpfxcC/2dSJwFtGV2hqi72R+D6vmE30314V1xhBV4Ls+0LZF1G6wkHQBUeXfkZmOJZ6kp4kqsf+U6Vj6QtJlwFlmdkyPCzs3wPkVhHPdkDRO0hGhGG4Poiu0+zIdl3Pp4E86Ote9PKI7vHYhuu3zTqKiJOcGPS9ics45l5AXMTnnnEtoUBUxjR492qZMmZLpMJxzbsB45ZVX1plZeaJ5gypBTJkyhXnz5mU6DOecGzAkrehqnhcxOeecS8gThHPOuYQ8QTjnnEvIE4RzzrmEPEE455xLKGUJQtLOkp5S1D3hQkn/EaaPVNSN4NLwnqgnNCSdrKi7wLclfTtVcTrnnEsslVcQbcDXzGwv4FDgS5KmA98GnjCzqcATYXwboQnnPxC1kjmdqK3+6SmM1TnnXCcpSxBmVmlR38GxpnsXE/W0dTpRf8iE9zMSrH4w8LaZLQtNKt8Z1nPOORfnsUVrue6Zd1Ky7bTUQUiaAhxA1D/B2NDGf6yt/zEJVpnAtl01rmLbbhzjt32xpHmS5lVXV+/QuJ1zrr97aEElt76wPCXbTnmCkFRC1E3hFWZWm+xqCaYlbFXQzG4wsxlmNqO8POHT4s45N2hVbmpi3PDClGw7pQkidCJ/D3Cbmd0bJq+VNC7MH0fUs1Vnq9i279+JRL1KOeeci7OmtomdhhWkZNupvItJwJ+BxWb267hZD7C1a8fzgfsTrD4XmCppF0l5RN0O9tS1oXPODSlmRsWmRsaVDbAEARxB1AfucZJeC69TifqLPUHSUuCEMI6k8ZJmAZhZG1Hn8I8QVW7fZWYLUxirc84NOJsaWmlu60hZEVPKWnM1s+dIXJcAcHyC5SuAU+PGZwGzUhOdc84NfJU1TQCMG2hFTM4551KrsqYR8AThnHOuk61XEAPwLibnnHOps6amiewsUV6an5Lte4JwzrkBqqKmkbGl+WRndVXdu308QTjn3AC1piZ1z0CAJwjnnBuw1tQ0paz+ATxBOOfcgGRmVNQ0puwOJvAE4ZxzA1JNYytNrR1exOScc25bqb7FFTxBOOfcgLTlIbnhfgXhnHMuTqqb2QBPEM45NyBVbooekhtT6gnCOedcnMqaJsak8CE58AThnHMD0praxpTewQSeIJxzbkCq3NTE+BTewQSeIJxzbsAxMypT3MwGeIJwzrkBp7axjcbW9pTewQQp7FFO0o3AR4AqM9snTPsbsEdYZDiwycz2T7DucqAOaAfazGxGquJ0zrmBpmJLR0GpLWJKKkFIygP2BAxYYmYtSax2M3ANcGtsgpl9Om6bvwJquln/WDNbl0x8zjk3lKwJz0CkuoipxwQh6cPAdcA7RH1M7yLpEjN7qLv1zOxZSVO62KaATwHH9Tpi55wb4tLxkBwkdwXxK6Kz+bcBJO0G/AvoNkH04ChgrZkt7WK+AY9KMuB6M7uhqw1Juhi4GGDSpEnbEZJzzg0MlTWNZAnGpKgnuZhkKqmrYskhWAZUbed+zwbu6Gb+EWZ2IHAK8CVJR3e1oJndYGYzzGxGeXn5doblnHP9X/SQXAE52am9zyiZK4iFkmYBdxGd2X8SmCvp4wBmdm9vdigpB/g48MGuljGzivBeJek+4GDg2d7sxznnBqvKmsaUNtIXk0z6KQDWAscAM4FqYCTwUaK7lHrrQ8CbZrYq0UxJxZJKY8PAicCCPuzHOecGpcqappTXP0ASVxBmdmFfNizpDqKEMlrSKuBKM/szcBadipckjQf+ZGanAmOB+6J6bHKA283s4b7E4Jxzg42ZsaamiZnTxqR8X8ncxXQTUdHSNszsou7WM7Ozu5h+QYJpFcCpYXgZsF9PcTnn3FBU29hGQ0s749NQxJRMHcSDccMFwMeAitSE45xzrjuVtdFDcql+BgKSK2K6J348FB09nrKInHPOdSldz0BA39pimgr4AwfOOZcBlZtS3xd1TDJ1EHVEdRAK72uAb6U4LueccwmsCQ/Jlaf4ITlIroipNOVROOecS0plTRPlpfnkpvghOegmQUg6sLsVzezVHR+Oc8657kTPQKS+eAm6v4L4VXgvAGYA84mKmfYFXgKOTG1ozjnnOqusaWTa2PQU7HR5jWJmx5rZscAK4MDQ3tEHgQOAt7tazznnXGrEepJL1xVEMoVYe5rZG7ERM1sA7J+yiJxzziVU2xQ9JJeOW1whuQflFkv6E/BXoruYzgUWpzQq55xz75OujoJikkkQFwKXAf8Rxp8F/piyiJxzziUU62o0Hc1sQHK3uTZJug6YZWZL0hCTc865BLZeQfSTOghJpwGvAQ+H8f0lPZDiuJxzznVSWdOE0tCTXEwyldRXEnXYswnAzF4DpqQsIueccwlVbmpkTJoekoPkEkSbmdWkPBLnnHPdWlPblLbiJUguQSyQ9BkgW9JUSVcDL6Q4Luecc51U1jQxriw9FdSQXIL4CrA30AzcDtQAV6QwJuecc52YGZWb0tMXdUyPCcLMGszsv4CZZnaQmX3PzJp6Wk/SjZKqJC2Im3aVpNWSXguvU7tY92RJSyS9LenbvfpEzjk3CNU1t1GfxofkILm7mA6XtIjwcJyk/SRdm8S2bwZOTjD9N2a2f3jNSrC/bOAPwCnAdOBsSdOT2J9zzg1aa2rS1w9ETDJFTL8BTgLWA5jZfODonlYys2eBDX2I6WDgbTNbZmYtwJ3A6X3YjnPODRoVm6KH5PrVFQSAma3sNKl9O/b5ZUmvhyKoEQnmTwDi97cqTEtI0sWS5kmaV11dvR1hOedc/5XuZjYguQSxUtLhgEnKk/R1+t4W0x+B3Yga+6tka5Pi8ZRgmnW1QTO7IbQ0O6O8vLyPYTnnXP8We0hubD+7i+lS4EtEZ/GriX7cv9SXnZnZWjNrN7MO4P+IipM6WwXsHDc+Eajoy/6cc26wqKxppLwkfQ/JQXJtMa0DztkRO5M0zswqw+jHgAUJFpsLTJW0C1FCOgv4zI7Yv3PODVRRPxDpu3qA5O5i2lXSPyVVh9tW75e0axLr3QG8COwhaZWkzwG/kPSGpNeBY4H/DMuOlzQLwMzagC8DjxAVZd1lZgv7/Amdc24QeG9DAxNHFKV1n8k093070W2nHwvjZwF3AId0t5KZnZ1g8p+7WLYCODVufBbwvltgnXNuKKptamXF+gY+NWPnnhfegZIpzJKZ/cXM2sIr1nGQc865NFhUUQvA9PFlad1vMlcQT4Wnme8kSgyfBv4laSSAmfXlWQfnnHNJWhgSxN79MEF8Orxf0mn6RUQJo8f6COecc323sKKG8tJ8xpSmt5I6mbuYdklHIM455xJbVFGb9qsH6KYOQtJBknaKGz8v3MH0+1jxknPOudRqam1nadVm9hk/LO377q6S+nqgBUDS0cDPgFuJmvu+IfWhOeecW7KmjvYOy8gVRHdFTNlxFdCfBm4ws3uAeyS9lvLInHPOxVVQ968riGxJsQRyPPBk3LxkKredc85tp4UVNZQW5LDzyPQ18x3T3Q/9HcAzktYBjcBsAEm7ExUzOeecS7GFoYJaStSOaWp1mSDM7CeSngDGAY+aWezhuCyibkidc86lUFt7B4srazn30MkZ2X+3RUVmNifBtLdSF45zzrmYZevqaW7ryEgFNSTZYZBzzrn0W1gRleZnooIaPEE451y/tXB1Lfk5WexWXpyR/XuCcM65fmphRS17jisjJ42dBMXrsg5CUh2JW20VYGaWmUIx55wbAsyMhRU1fGS/8RmLobu7mErTGYhzzrmtVm1spLapLWMV1NCLB94kjQG2NCVoZu+lJCLnnHMZr6CG5LocPU3SUuBd4BlgOfBQEuvdGLooXRA37ZeS3pT0uqT7JA3vYt3loWvS1yTNS/bDOOfcYLGwopbsLLHnTpkrzEmm5uNHwKHAW6Hp7+OB55NY72bg5E7THgP2MbN9gbeA73Sz/rFmtr+ZzUhiX845N6gsrKhl9/ISCnKzMxZDMgmi1czWA1mSsszsKWD/nlYys2eBDZ2mPWpmbWF0DjCxl/E659yQsGB1TUbrHyC5OohNkkqAZ4HbJFUBbT2sk4yLgL91Mc+ARyUZcL2Zddm8uKSLgYsBJk2atAPCcs65zKqua6aqrjntfVB3lswVxOlAA/CfwMPAO8BHt2enkv6LKMnc1sUiR5jZgcApwJdCfxQJmdkNZjbDzGaUl5dvT1jOOdcv9IcKakguQYwB8syszcxuAf4P6HOtiaTzgY8A58Q1ALgNM6sI71XAfcDBfd2fc84NNLE+IAbCFcTdQEfceHuY1muSTga+BZxmZg1dLFMsqTQ2DJwILEi0rHPODUYLK2qYNLKIYYW5GY0jmQSRY2YtsZEwnNfTSpLuAF4E9pC0StLngGuIrj4eC7ewXheWHS9pVlh1LPCcpPnAy8C/zOzhXn0q55wbwGJ9QGRaMpXU1ZJOM7MHACSdDqzraSUzOzvB5D93sWwFcGoYXgbsl0Rczjk36NQ2tbJifQOf/GDmb/JMJkFcSnT30jVE7TCtBM5LaVTOOTdELY71QT0hsxXUkESCMLN3gEPDra4ys7rUh+Wcc0NTrIK6XxcxSTrXzP4q6audpgNgZr9OcWzOOTfkLKioobw0nzGlBT0vnGLdXUHEeqhIdEtrwttTnXPObZ9F/aSCGrpv7vv6MPi4mW3T9pKkI1IalXPODUH1zW28XbWZD+01NtOhAMnd5np1ktOcc85th+feXkdbh3H47qMyHQrQfR3EYcDhQHmneogyIHPNCzrn3CD15OIqSvNzOGjKyEyHAnRfB5EHlIRl4ushaoEzUxmUc84NNR0dxlNLqjh6Wjm5GeqDurPu6iCeAZ6RdLOZrZBUbGb1aYzNOeeGjIUVtVTVNXPcnmMyHcoWyaSp8ZIWAYsBJO0n6drUhuWcc0PLE2+uRYKZe/SfVqmTSRC/BU4C1gOY2Xygy+a3nXPO9d5Tb1ax/87DGVWSn+lQtkiqoMvMVnaa1J6CWJxzbkiqqmti/qoaju9HxUuQXFtMKyUdDpikPOByQnGTc8657ff0kmoAjtuzfzz/EJPMFcSlwJeACcAqov6ov5TCmJxzbkh5cnEV44YVsNe4PvfFlhLdXkFIygZ+a2bnpCke55wbUprb2pm9tJrTD5iwpa27/qLbKwgzayd6UK7HDoKcc8713tx3N1Lf0t7v6h8guTqI5cDzkh4AtjwH4a25Oufc9nvizbXk52Rx+G6jMx3K+yRTB1EBPBiWLY17dUvSjZKqJC2ImzZS0mOSlob3EV2se7KkJZLelvTt5D6Kc84NLGbGk29WcfhuoyjM638tGCXTYdAPACSVRqO2Oclt30zUB/WtcdO+DTxhZj8LP/zfBr4Vv1Ko9/gDcAJRpfhcSQ+Y2aIk9+uccwPCsnX1rFjfwOeP2jXToSTU4xWEpH0k/RtYACyU9IqkvXtaz8yeBTZ0mnw6cEsYvgU4I8GqBwNvm9kyM2sB7gzrOefcoPLk4iqAftW8RrxkiphuAL5qZpPNbDLwNeD/+ri/sWZWCRDeEx2VCUT9XsesCtOcc25QefLNKvbcqZQJwwszHUpCySSIYjN7KjZiZk+ztbe5VEh0n1eXPdhJuljSPEnzqqurUxiWc87tODWNrcxdvoFj++nVAySXIJZJ+m9JU8Lre8C7fdzfWknjAMJ7VYJlVgE7x41PJKooT8jMbjCzGWY2o7y8/zRy5Zxz3Zm9tJq2DuuXt7fGJJMgLgLKgXvDazRwYR/39wBwfhg+H7g/wTJzgamSdgnPX5wV1nPOuUHjyTerGF6UywGTEt7M2S8kcxfTRuBySSW9uIMJSXcAM4HRklYBVwI/A+6S9DngPeCTYdnxwJ/M7FQza5P0ZeARop7rbjSzhb38XM4512+1dxhPL6lm5rRysrP619PT8XpMEKGhvj8R9S43SdJ+wCVm9sXu1jOzs7uYdXyCZSuAU+PGZwGzeorNOecGornLN7ChvqVf1z9AckVMv8H7g3DOuR3mr3NWUFaQwwnT+1frrZ15fxDOOZdGVbVNPLxgDZ+asTNFecm0dpQ53h+Ec86l0e0vv0e7GeceOjnTofTI+4Nwzrk0aWnr4LaX3uOYaeVMGZ3Kx8l2jGTuYloHeH8Qzjm3nR5ZuIbqumbOP2xKpkNJSjJ3MZUDXwCmxC9vZhelLiznnBt8bn1xOZNGFnHMtIHxUG8ydRD3A7OBx/HKaeec65NFFbXMXb6R/zp1L7L68bMP8ZJJEEVm9q2eF3POOdeVv8xZTkFuFp+cMTHToSQtmUrqByWd2vNizjnnEqlpaOW+f6/mjP0nMLxo4PTg3OUVhKQ6olZUBXxXUjPQGsbNzMrSE6Jzzg1sd7+ykqbWDj57WP+/tTVelwnCzHrsVtQ551z3OjqMv8xZwYzJI9h7/LBMh9MryfQo90Qy05xzzr3fM0urWbG+gfMOn5LpUHqtuyKmAqKOgUZLGsHWjnzKgPFpiM055wa8W19YzuiSfE7ee6dMh9Jr3d3FdAlwBVEyeIWtCaIW+ENqw3LOuYFvxfp6nn6rmq8cN5W8nKSavutXuquD+B3wO0lfMbOr0xiTc84NCtc+9Q45WeIzB0/KdCh90mNK8+TgnHO9t2B1DXe9spLzD5vCTsMKMh1Onwy8ax7nnOvnzIwf/2sRwwtz+crxUzMdTp+lPUFI2kPSa3GvWklXdFpmpqSauGW+n+44nXOurx5dtJY5yzbw1ROmMawwN9Ph9FlSvVVI2pf3N9Z3b192aGZLiJoMR1I2sBq4L8Gis83sI33Zh3POZUpzWzv/M2sxU8eUcPYArXuISaY11xuBfYGFQEeYbECfEkQnxwPvmNmKHbAt55zLuFtfWMGK9Q3cfOFB5GQP7FL8ZK4gDjWz6Sna/1nAHV3MO0zSfKAC+LqZLUxRDM45t0Os39zM759cysw9ypm5x5hMh7PdkklvL0ra4QkidF96GnB3gtmvApPNbD/gauAf3WznYknzJM2rrq7e0WE651zSfvP4WzS0tPO9D++V6VB2iGQSxC1ESWKJpNclvSHp9R2w71OAV81sbecZZlZrZpvD8CwgV9LoRBsxsxvMbIaZzSgvHxidcDjnBp+31tZx+0vvce4hk9h9zOBoyi6ZIqYbgc8Cb7C1DmJHOJsuipck7QSsNTOTdDBRIlu/A/ftnHM7jJnxowcXUZKfwxUfmpbpcHaYZBLEe2b2wI7cqaQi4ASi5jxi0y4FMLPrgDOByyS1AY3AWWZmOzIG55zbUZ5eUs3spev4749MZ0TxwOnvoSfJJIg3Jd0O/BNojk3s622uYd0GYFSnadfFDV8DXNPX7TvnXLpsamjhv+57g11HF/PZQwdWfw89SSZBFBIlhhPjpu2o21ydc27AMjO+8ffXqd7czD2XHT4gG+TrTo8JwswuTEcgzjk30NzywnIeW7SW7314L/adODzT4exwyTwodxPRFcM2zOyilETknHMDwILVNfzPrDc5bs8xfO7IXTIdTkokU8T0YNxwAfAxoofXnHNuSNrc3MZX7vg3I4pz+d9P7oeknlcagJIpYronflzSHcDjKYvIOef6ue//YwEr1tdz+xcOZeQgumups77UqEwFBnYLVM4510d/f2UV9/57NZcfP5VDdx3V8woDWDJ1EHVEdRAK72uAb6U4Luec63fertrMf/9jAYfsMpKvHDdw+3lIVjJFTIPjmXHnnNsONQ2tfPG2VyjMy+Z3Zx1AdtbgrHeIl/b+IJxzbqBpaGnjolvmsnxdAzddeNCA7UK0tzLdH4RzzvVrLW0dXPrXV/n3exu59pwDOWL3hO2GDkqZ7g/COef6rfYO46t3vcazb1Xz8098gJP3GZfpkNIqY/1BOOdcf2ZmfP/+BTz4eiXfOWVPPn3Q0Lt5M5kriFh/EGuI2mQSYGa2b0ojc865DPrfR5dw20vvcdnM3bjkmN0yHU5GZLI/COec65f+NHsZf3jqHc4+eBLfPGmPTIeTMRnpD8I55/ojM+N3Tyzlt48v5cMfGMePz9hn0DajkYyM9AfhnHP9TWt7B9+99w3ufmUVnzhwIj/9+AeGxLMO3fH+IJxzQ15dUytfvO1VZi9dx+XHT+U/PzR1SF85xHh/EM65IW1NTRMX3jyXt9bW8fNPfGBI3q3UlS4ThKRvmtkvJF1N4v4gLu/rTiUtB+qAdqDNzGZ0mi/gd8CpQANwgZm92tf9OedcIkvW1HHhTS9T09jKjRccxDHTyjMdUr/S3RXE4vA+L0X7PtbM1nUx7xSiVmOnAocAfwzvzjm3Qzy6cA1fu3s+hbnZ/O2Sw9hnwrBMh9TvdJkgzOyfYfBvZtYUP09Sqp81Px241cwMmCNpuKRxZlaZ4v065wa5ptZ2/mfWYm59cQV7jy/jhvNmMGF4YabD6peSeZL6ZUmHxkYkfQJ4YTv3a8Cjkl6RdHGC+ROAlXHjq8K095F0saR5kuZVV1dvZ1jOucFs6do6zvjD89z64go+f+Qu3PvFwz05dCOZu5jOAW6U9DQwHhgFHLed+z3CzCokjQEek/SmmT0bNz/R7QPvqwcBMLMbgBsAZsyYkXAZ59zQZmbcOXclP/jnQorzcrjpgoM4ds8xmQ6r30vmLqY3JP0E+AtRxfLRZrZqe3ZqZhXhvUrSfcDBQHyCWAXsHDc+Ee8H2znXBzUNrXz3vjf41xuVHLn7aH79qf0YUzY0muveXsk09/1nYDeiJr+nAf+UdI2Z/aEvO5RUDGSZWV0YPhH4YafFHgC+LOlOosrpGq9/cM71hplx76ur+Z9Zi6lpbOXbp+zJxUftStYQf/itN5IpYloAfD5UGL8b6iN+vR37HAvcFx5CyQFuN7OHJV0KYGbXAbOIbnF9m+g2V38WwzmXtKVr6/jePxbw0rsbOGDScH58xj7sPd7vUuotRb/7g8OMGTNs3rxU3ZXrnOvvGlva+f2TS/m/Z5dRnJ/Dt07ek7MO2tmvGroh6ZXOz6LFJFPENBX4KTAd2FJwZ2a77rAInXNuO5gZjyxcw48eXMzqTY2c+cGJfOeUPRlVkp/p0Aa0ZIqYbgKuBH4DHEtU3OPp2DmXcWbGc2+v438fWcL8VTVMG1vCXZccxsG7jMx0aINCUo31mdkTkmRmK4CrJM0mShrOOZcRr6zYyC8feZM5yzYwYXghvzhzXz5+wARyspN5vMslI5kE0SQpC1gq6cvAasBvIHbOZcSiilp+/dgSHl9cxeiSPK766HTOPmQS+TnZmQ5t0EkmQVwBFAGXAz8iekju/BTG5Jxz2zAznn97PTfMXsazb1VTVpDDN07agwsOn0JxfjI/Y64vknlQbm4Y3IzfbuqcS6PW9g4efL2CG559l8WVtYwuyecbJ+3BuYdMZlhRbqbDG/S6a+67225Gzey0HR+Oc87BhvoW7p63kpueX86a2iamjinhF2fuy+n7j/eipDTq7griMKIG8+4AXsLvXHLOpVBHh/HisvXc8fJ7PLJwDa3txuG7jeKnn/gAx0wt92cZMqC7BLETcAJwNvAZ4F/AHWa2MB2BOeeGhqraJu5+ZRV/m7uS9zY0MKwwl3MPncxZB01ij51KMx3ekNZdfxDtwMPAw5LyiRLF05J+aGZXpytA59zgU9fUyqML1/LA/Aqee3sd7R3GIbuM5GsnTuOkvXeiINeLkfqDbiupQ2L4MFFymAL8Hrg39WE55wabptZ2nl5SxQPzK3hicRXNbR1MGF7IF47alU/NmMiu5SWZDtF10l0l9S3APsBDwA/MbEHaonLODQp1Ta0881Y1jy1ay5OLq6hrbmN0SR5nHbQzp+0/ngMnjSA03On6oe6uID4L1BM18X153B9RgJlZWYpjc84NQGtqmnhs8VoeW7SWOe+sp6W9g5HFeZy8z06ctv94Dtt1lD/tPEB0Vwfhf0HnXI9a2jp49b2NPPtWNc8urWbB6loApowq4vzDJ3PC9J344OQRZPtdSAOOP4LonOsVM+PddfXMXrqO2UurefGd9dS3tJOTJQ6cNIJvnLQHJ04fy+5jSrz4aIDzBOGc65aZ8U51PXOWreeldzfw0rL1VNU1AzB5VBEfP3AiR00dzWG7jaK0wJ9uHkw8QTjnttHS1sHCihpefW8Tr67YyEvvbmDd5ighjC3L57DdRnHILqM4YvdRTB5VnOFoXSqlPUFI2hm4lehBvA7gBjP7XadlZgL3A++GSfeaWed+q51z28nMWFPbxPyVm7YkhNdX19DS1gHAhOGFHDV1NIfsMpJDdx3F5FFFXmw0hGTiCqIN+JqZvSqpFHhF0mNmtqjTcrPN7CMZiM+5Qauqrok3VtXw+qoa3lgdvceuDvKys9hnQhnnHTqZAyeP4MBJI9hpWEEPW3SDWdoThJlVApVhuE7SYmAC0DlBOOf6qL3DeHfdZhZV1rG4spZFFbUsrqzdUncgwdQxJRw9bTT7ThjGByYOZ58JZd4QnttGRusgJE0BDiBqDLCzwyTNByqAr3fVBpSki4GLASZNmpSiSJ3rn8yMypom3lpbF16btww3tUbFRDlZYurYUo6cOprp48rYb+fhTB9X5v0ouB7JzDKzY6kEeAb4iZnd22leGdBhZpslnQr8zsym9rTNGTNm2Lx581ITsHMZ1NrewXsbGlhWXc871Zt5p2oz71RvZunazdQ1t21Zrrw0n2ljS9hjbBnTx5cxfVwZu48pIS/HH2tyiUl6xcxmJJqXkVMISbnAPcBtnZMDgJnVxg3PknStpNFmti6dcTqXTh0dRmVtE8vX1fPuunqWr6tn+fpoeMX6Bto6tp7MlZfms+voYs44YALTxpYwbWwp08aWMqI4L4OfwA02mbiLScCfgcVm9usultkJWGtmJulgIAtYn8YwnUuJ+uY2Vm1s5L0NDby3oYGV4T32it09BJCfk8WUUcXsPqaEk/beid3KS9htTAm7lhdT5s8buDTIxBXEEUTtPL0h6bUw7bvAJAAzuw44E7hMUhvQCJxlmSoLcy5JZsbGhlYqNjVSWdPE6o0NrNrYyKqNjaze1MiqjQ1sbGjdZp2S/BwmjSxi9/ISjttzDFNGFTNldBFTRhWzU1mBd5LjMioTdzE9Rw+905nZNcA16YnIuZ6ZGRvqW1hT28SamibW1DaxtqaJyvCq2NRIRU3jlorhmILcLCYML2TiiCI+MHEYE0dEw5NHFjFpZBHDi3L9uQLXb/ltDG5Ia++Ifvir65qp3txMdV0za2ubqKptoio2XNdMVW0zLe3b/vhnKaoLGD+8kL3GlXHcnmMYP7yQ8cMLGDeskAkjChlVnOcJwA1YniDcoNPS1sGG+hbWbW5mfX0L6zc3R8ObW6iOvYeEsH5zMx0JCi9LC3IYW1bA2LJ8DpoykjGl+ew0rIBxwwoYWxYlgNEled5stRvUPEG4fq29w6htbGVjQwsbG1rZWN/ChoaWbd/rW9lQHyWDDZtbtrntM15eThblJfmMLsljp2EF7DtxGOWl+dGrJH/L8JjSAgrz/IEx5zxBuLRobmunprGV2sZWauJemxqiVzTcQk1jKxsbouGNDa3UNrXS1e0JedlZjCzOY0RxHqOK85g4ooiRxXmMLsljZHH+luHRJfmMKsmjJD/Hi3uc6wVPEK5HHR1GQ2s7m5vaqGtqpbapjdqmVupi442x6dG02sawTGM0raax9X2Vt52VFeQwvCiPYYW5DC/KZeeRRYwoymV4UR7DC3MZURwNjyrOY0RRHiOL8yjKy/YffOdSyBPEIGVmNLa2U9/cTn1zG5ub26hvbqO+pY26prZtpm9ubmNzU/Re19zG5qbWLdPqwvyebjLOyRJlhbmUFeSE91zGluVTmp/LsKJchhXmUlYY3gtyGBaGRxTlUVaY672NOdcPeYLIMDOjqbWDhpY2GlraaWxtp6GlnYaWNhpb2qlvaacxzItNr28O7y3tNDSH95Y2Gprb2dwcLVvf0vOPekxxXjYlBTmU5IdXQQ7lpfmUFuRSkp9DWUE0LTZeuiUJRNPKCnIpyM3ys3nnBhlPEF3o6DCa2zpoam2nqa2dptYw3Br9iDeH8cbwis1vbImbFjfc0BKt29ASLbNluLW9V3HlZInCvGxK8nMoysumKC96H1NaQNGo2PQcSvKzKcrPoTg/h+K8bIrDj3/0Ho1H83L87N05l5AnCODDv59NXVPblgTQ1NaxTZMHvZElKMrLoSA3m8K8LApzsynMzSY/N5uRxXlMGJ5NYV40rSi8F4Yf+cK87PCjn01hbjStOH9rEijKy/FG15xzaeMJApg2thQzoyA3m4LcbPJzs8jPyaYgN4uCnOiHuyA3a8sPfUGYF/uhj00vzM0mN1te1OKcGxQ8QQC/+fT+mQ7BOef6HS+vcM45l5AnCOeccwl5gnDOOZeQJwjnnHMJeYJwzjmXkCcI55xzCXmCcM45l5AnCOeccwnJkm3RbQCQVA2syHQc/cBoYF2mg+gn/Fhsy4/HVn4sIpPNrDzRjEGVIFxE0jwzm5HpOPoDPxbb8uOxlR+LnnkRk3POuYQ8QTjnnEvIE8TgdEOmA+hH/Fhsy4/HVn4seuB1EM455xLyKwjnnHMJeYJwzjmXkCeIAUrSyZKWSHpb0rcTzD9H0uvh9YKk/TIRZ7r0dDziljtIUrukM9MZXzolcywkzZT0mqSFkp5Jd4zplMT/yjBJ/5Q0PxyPCzMRZ79kZv4aYC8gG3gH2BXIA+YD0zstczgwIgyfAryU6bgzeTzilnsSmAWcmem4M/jdGA4sAiaF8TGZjjvDx+O7wM/DcDmwAcjLdOz94eVXEAPTwcDbZrbMzFqAO4HT4xcwsxfMbGMYnQNMTHOM6dTj8Qi+AtwDVKUzuDRL5lh8BrjXzN4DMLOhfjwMKFXUmXwJUYJoS2+Y/ZMniIFpArAybnxVmNaVzwEPpTSizOrxeEiaAHwMuC6NcWVCMt+NacAISU9LekXSeWmLLv2SOR7XAHsBFcAbwH+YWUd6wuvfcjIdgOsTJZiW8H5lSccSJYgjUxpRZiVzPH4LfMvM2qMTxUErmWORA3wQOB4oBF6UNMfM3kp1cBmQzPE4CXgNOA7YDXhM0mwzq01xbP2eJ4iBaRWwc9z4RKKzn21I2hf4E3CKma1PU2yZkMzxmAHcGZLDaOBUSW1m9o+0RJg+yRyLVcA6M6sH6iU9C+wHDMYEkczxuBD4mUWVEG9LehfYE3g5PSH2X17ENDDNBaZK2kVSHnAW8ED8ApImAfcCnx2kZ4bxejweZraLmU0xsynA34EvDsLkAEkcC+B+4ChJOZKKgEOAxWmOM12SOR7vEV1NIWkssAewLK1R9lN+BTEAmVmbpC8DjxDdpXGjmS2UdGmYfx3wfWAUcG04a26zQdpyZZLHY0hI5liY2WJJDwOvAx3An8xsQeaiTp0kvxs/Am6W9AZRkdS3zMybAceb2nDOOdcFL2JyzjmXkCcI55xzCXmCcM45l5AnCOeccwl5gnDOOZeQJ4ghIrRg+pqkBZLuDve/p3P/3+00/kKK97dn+Lz/lrTbDtzuckmje7H8FfHHWtLmXu5vpqTDu5h3Wnct13ZadoqkxnBMFkm6TtJ2//+H+B4MwxdIuqaH5bcsI+nSvjTzIekqSV9PMH2KpEF5u26meIIYOhrNbH8z2wdoAS6NnykpOxU7VSSLqMXMLcws4Y/eDnQGcL+ZHWBm76R4X925AtieZDyTqGXe9zGzB8zsZ73Y1jtmtj+wLzCd6Bj1SFJKnpcKz2Tcmoptux3DE8TQNBvYPZz9PSXpduANSQWSbpL0RjjzPha2nPXdL+nh0K7+lbENSfpquCpZIOmKMG2KpMWSrgVeBf4MFIaz19vCMpvDuyT9Mqz/hqRPh+kzQ2Nyf5f0pqTblKARJUn7S5qjqN+L+ySNkHQq0Q/z5yU9lWCdkyW9qqj9/yfCtJGS/hG2M0dRMyVIGiXp0XA8rieubR9J50p6OXyu6zsnWUmXA+OBp+LjkPSTsO85ip7cRdJHJb0U9vO4pLGSphAl8v8M+ziq0/bjz8Y/GY7hfEVNZ3TJzNqAF8J34AuS5ob17old7Ui6WdKvQ9w/l3Swon5F/h3e9+huH5LKw/bmhtcRCZa5StLXJY0Pny/2apc0OdExiVt9P0lPSloq6QsJtp0dvldzw9/0ku7idV3IdHvj/krPC9gc3nOImlq4jOjstB7YJcz7GnBTGN6TqAmCAuACoJLoyexCYAFR20YfJGr9spiomeSFwAHAFKIndA/tvP8E8XwCeIzoKdexYZ/jQmw1RG3nZAEvAkcm+FyvA8eE4R8Cvw3DVwFfT7B8OVHrnrHPPDK8Xw1cGYaPA14Lw78Hvh+GP0zU0NtootY//wnkhnnXAucl2N9yYHTcuAEfDcO/AL4Xhkew9cHVzwO/6u5zhHkXANeE4TeACWF4eIJlpwALwnARURMUpwCj4pb5MfCVMHwz8CCQHcbLgJww/CHgnjA8E3gwQTy3x/5ewCRgcYJl3vfZgC8BdyVxTOYTfRdHh7/n+E6f8eK4Y5sPzIv9zf2V/Mub2hg6CiW9FoZnE53VHw68bGbvhulHEv1QYmZvSlpB1DQ0wGMWGvyTdG9Y1oD7LGr0LTb9KKK2blaY2Zwk4joSuMPM2oG1ino3OwioDbGtCtt+jegH4LnYipKGEf0YxnpEuwW4u4f9HQo8G/vMZrYhLo5PhGlPhiuHYcDRwMfD9H9JivWxcTxRgpwbLmwKSa6fiRaiH16AV4ATwvBE4G+SxhF1bPNugnW78zxRcxF3EbXBlchu4TgaUfHbQ5KOkfRjok6ESoiapIi5O/xdAIYBt0iaGtbP7SGeDwHT4y76yiSVdrdCuMr4PNF3CLo/JvebWSPQGK5yDiZqkTXmRGBfbe05cBgwld4f1yHNE8TQ0WhR+fMW4Z+3Pn5SN+t3bpPFeli+vpt524TRzbzmuOF2dsz3VSRuGr27ZqG7Wv4WM/tOL/ffauG0lm0/09XAr83sAUkzic6Sk2Zml0o6hOgq5zVJ+9v7W/B9p/N3gOhK4Qwzmy/pAqIrgpj4v+GPgKfM7GOh6OvpHkLKAg4LP+JbJCgljE0fR3TScpqZxSryuzsmib6P22yS6GroEVyfeR2Ei/cscA6ApGlERQNLwrwTQjl9IVHl5vNh+TMkFUkqJuqQZ3YX226VlOis81ng06HMuJzojD2pZpbNrAbYGFc2/1mgp/6VXwSOkbRL+Jwj4+KIffaZRM1h13aafgpRsQfAE8CZksbEtiNpcoL91QHdnjkHw4DVYfj83q4vaTcze8nMvg+sY9smrrtTClSGv805ScZ3QRLbfRT4clx8+3e1YNj3XUSN5MW3PNzVMQE4XVGd2SiipDa30/xHgMti3zlJ08J31PWCJwgX71ogW1Grln8DLjCz2Fn8c8BfiC7j7zGzeWb2KtEZ6MvAS0Stgv67i23fALyuUEkd5z6ieoT5RP1Ff9PM1vQi5vOBX0p6HdifqB6iS2ZWTVQ+fa+k+eFzQnR2OiNs52ds/UH6AXC0pFeJii1i3XQuAr4HPBrWeYyo7qSzG4CHlKCyvJOrgLslzSb6gY/5J/CxRJXUnfxSUSX/AqKkNr+H/cX8N9Hf7jHgzW6W+wXwU0nPE9UX9eRywvGUtIhOd811cjhRseIP4iqqx9P1MYHoO/cvou50f2Rmnft4+BNRv9uvhmNyPV5i0mvemqvrUSh6mGFmX+5pWefc4OFXEM455xLyKwjnnHMJ+RWEc865hDxBOOecS8gThHPOuYQ8QTjnnEvIE4RzzrmE/h/f1036tiwrzwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "P = np.linspace(0.05,0.95)\n",
    "\n",
    "speedup = lambda p: 1/(1-p)\n",
    "\n",
    "S = np.vectorize(speedup)(P)\n",
    "\n",
    "f1=plt.figure()\n",
    "plt.plot(P,S)\n",
    "plt.xlabel(\"Proportion of code that is Parallelizable\")\n",
    "plt.ylabel(\"Maximum theoretical Speedup\")\n",
    "plt.title( \"Speedup vs. Algorithmic Parallelism\")\n",
    "\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "204608a1",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "What is Amdahl's Law?\n",
    "---------------------------------\n",
    "\n",
    "* Speedup a program can achieve is limited by the level of parallelization of its design.\n",
    "\n",
    "* If code is 95% parallelized, then 5%, or 1 part in 20, is serial... that means, no matter how many processors are put on the job, eventually you are limited by the part that isn't parallel. It's asymptotic speedup (that it will never reach, is 20x.)\n",
    "\n",
    "* The more processors you want to use, the more this law bites.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4be08b2a",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": [
     "hide-input",
     "hide_code"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEaCAYAAAAL7cBuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABG0ElEQVR4nO3dd3xUZdbA8d9Jb9SEQKhBiYAKIiLoioooFmxrQVbFsioIigVFxbIurquigoqiFCu69rWAvoi4CoJdQBAUUJBIC0kIkEBCysyc9487CZMwCRPIZFLO9+N85pbn3nuGjPfMbecRVcUYY4ypKCzUARhjjKmbLEEYY4zxyxKEMcYYvyxBGGOM8csShDHGGL8sQRhjjPHLEoQxARKRV0Tk3wG2XSAi11UyL1VEVEQiDiKWASKy6UCXNyYQliBMg+PdOe8QkehQxxIIb7wDQh2HMRVZgjANioikAicCCpwX2miMqd8sQZiG5krgO+AV4CrfGd5TRM+JyCcisltEvhaRNiLylPeIY7WIHO3T/mgRWSoiu0TkbSDGZ14LEflYRLK9y34sIu0rxNLJu41dIjJPRJL2F7yI9BWRxSKSJyKZIvJEIB9aRMaJyDrvtn4VkQt85v0pIsd4h4d5T28d7h2/TkQ+DGQbpvGxBGEamiuB172vM0SkdYX5lwD3AUlAEfAtsNQ7/l/gCQARiQI+BF4DWgLvAhf5rCcMeBnoBHQE9gBTKmzrMuDvQDIQBYz1F7CqDlDVBd7RycBkVW0KHAq8E+DnXodz5NQMeAD4j4ikeOd9CQzwDp8E/AGc7DP+ZYDbMI2MJQjTYIhIf5wd9juqugRnp3lZhWYfqOoSVS0EPgAKVfVVVXUDbwOlRxDHAZHAU6paoqr/BX4sXYmq5qjqe6paoKq7gIfYu9Mt9bKq/qaqe3B29L0C+BglQBcRSVLV3ar6XSCfXVXfVdUtqupR1beB34G+3tlf+sR2IvCIz/jJWIIwlbAEYRqSq4B5qrrNO/4GFU4zAZk+w3v8jCd4h9sCm7V8Ncs/SwdEJE5EpntP3+QBC4HmIhLu036rz3CBz7qrci1wGLBaRH4UkXMCWAYRuVJElonIThHZCRyJc1QETgI4UUTaAOE4ifAE7/WaZsCyQLZhGp8Dvs3OmLpERGJxTh+Fi0jpjjkaZ6d9lKour+YqM4B2IiI+SaIjzlEJwO1AV6Cfqm4VkV7AT4AczOdQ1d+BS0UkDLgQ+K+IJKpqfmXLiEgn4HngVOBbVXWLyLLSWFR1rYgUADcDC1V1l/ffaATwlap6DiZm03DZEYRpKP4KuIHDcU7l9AK6A4twrktU17eAC7hZRCJE5EL2nrIBaIJzxLFTRFoC/zzQwH15LyK38u60d3onu/ezWDzOXVvZ3nX8HecIwteXwGj2nk5aUGHcmH1YgjANxVU45/w3qOrW0hfOhePLq/tQmqoW4/yCvxrYAQwF3vdp8hQQC2zDuWtq7kF/AseZwC8ishvngvXfvNdLqor1V2ASTlLLBHoAX1do9iVOUltYybgx+xDrMMgYY4w/dgRhjDHGL0sQxhhj/LIEYYwxxi9LEMYYY/yyBGGMMcavBvWgXFJSkqampoY6DGOMqTeWLFmyTVVb+ZvXoBJEamoqixcvDnUYxhhTb4jIn5XNs1NMxhhj/LIEYYwxxi9LEMYYY/xqUNcg/CkpKWHTpk0UFlZZzsY0EDExMbRv357IyMhQh2JMvRe0BCEiHYBXgTaAB5ihqpO9lS/fBlKBdOASVd3hZ/kzcYqVhQMvqOqEA4lj06ZNNGnShNTUVEQOqhKzqeNUlZycHDZt2kTnzp1DHY4x9V4wTzG5gNtVtTtO71w3evvBHQd8rqppwOfe8XK8na48C5yFU7750tI+dKursLCQxMRESw6NgIiQmJhoR4vG1JCgHUGoagZOpyt4OyhZBbQDzmdv/7gzcerS31Vh8b7AWlX9A0BE3vIu9+uBxGLJofGwv7U5EOrxgNuNut3Oe+m4x4O6XFA6ruoMezzOcOnL40E9CpQOe5weOtRpi2ol873TSqtqq3rXy95p7N1O2Ta9k0vnSVQUCSeeWOP/LrVyDcLbteHRwPdAa2/yQFUzRCTZzyLtgI0+45uAfpWsewROz1h07NixBqOuOampqTRp0oTw8HAiIiLKntXYvn07Q4cOJT09ndTUVN555x1atGjB119/zahRo4iOjubNN9+kS5cu7Ny5k6FDhzJ37txa3Qmmp6dzzjnnsHLlShYsWMDEiRP5+OOPK20fSJvqCsY6TeioKlpcjCc/v/yroGDv8J5CtKQELS4u/+5v2v7eXS5nh1z67psAvAmhvgtPSuKwrxbV+HqDniBEJAF4D7hVVfMC3Ln5a+S34wpVnQHMAOjTp0+d7dxi/vz5JCUllZs2YcIETj31VMaNG8eECROYMGECjz76KJMmTeK9994jPT2dqVOnMmnSJB588EHuueeeoCQHl8tFRESDv1/B1CAtKcG1bRuu7GznlZWFOze33A7fvc+Ov6BsHJerehuMiEAiI5GoKO97JBIZSVhUFERGEhbpTA+Li0WaNfPOd6YRGYGERyDhYRAWvvc9Irz8eHgYVGwXHo6Eh0NYmPMuYc50ERDnXcIEwsIAgTBBwnzn44z7m49423mPfEv/3y4blrI9Ydl833llbUEignNTRlD3CiISiZMcXlfV0t64MkUkxXv0kAJk+Vl0E9DBZ7w9sCWYsYbCrFmzWLBgAQBXXXUVAwYM4NFHHyUyMpI9e/ZQUFBAZGQk69atY/PmzZx88smVris1NZWhQ4cyf/58AN544w26dOnCRx99xL///W+Ki4tJTEzk9ddfp3Xr1owfP54tW7aQnp5OUlISDz/8MFdccQX5+U7Xx1OmTOEvf/lLpdvLz8/npptuYsWKFbhcLsaPH8/5559faXu3281dd93Fp59+iogwfPhwbrrpJj7//HPGjh2Ly+Xi2GOPZerUqURHRzN37lxuvfVWkpKS6N279wFv11SPp7gYd3Y2JVlZPjv/7HKJwJWdjXvHDp9TIHtJZCRh8fHOKy6OsPh4whOaENm6Tblp5V5l07zt4+ORmBgnGURFIRERzs7Z1Lpg3sUkwIvAKlV9wmfWbJzuISd432f5WfxHIE1EOgObgb8Blx1sTA989Au/bsk72NWUc3jbpvzz3COqbCMinH766YgI119/PSNGjAAgMzOTlJQUAFJSUsjKcnLl3XffzYgRI4iNjeW1115j7NixPPjgg/uNpWnTpvzwww+8+uqr3HrrrXz88cf079+f7777DhHhhRde4LHHHmPSpEkALFmyhK+++orY2FgKCgr47LPPiImJ4ffff+fSSy+tsmzJQw89xMCBA3nppZfYuXMnffv25bTTTqu0/YwZM1i/fj0//fQTERERbN++ncLCQq6++mo+//xzDjvsMK688kqmTp3KyJEjGT58OF988QVdunRh6NCh+91ufHz8fv99jHN6x5WZSeGvqyhc9SslGzZ6d/5ZuLKycefm7rtQeDgRSUlEtGpFZNu2xPbqRUSrVntfycnOe4vmSFRU7X8oEzTBPII4AbgCWCEiy7zT7sFJDO+IyLXABmAIgIi0xbmddbCqukRkNPApzm2uL6nqL0GMNai+/vpr2rZtS1ZWFoMGDaJbt26cdNJJlbbv1asX3333HQALFy6kbdu2qCpDhw4lMjKSSZMm0bp1632Wu/TSS8vex4wZAzi3+Q4dOpSMjAyKi4vL3f553nnnERsbCzjPi4wePZply5YRHh7Ob7/9VuVnmjdvHrNnz2bixImAc7fYhg0bKm3/v//9j5EjR5adymrZsiXLly+nc+fOHHbYYYBzFPXss88yYMAAOnfuTFpaGgDDhg1jxowZVW63e/fuVcbbGKnHQ/Gff1K0ahWFq1ZR+MuvFK5a5fz6BxAhok0bIpJbEdmpE7F9+hBZurP32fGHt2hhv+AbqWDexfQV/q8lAJzqp/0WYLDP+BxgTk3GtL9f+sHStm1bAJKTk7ngggv44YcfOOmkk2jdujUZGRmkpKSQkZFBcnL56/Wqyr///W/efvttRo8ezQMPPEB6ejpPP/00Dz300D7b8b0+UTp80003cdttt3HeeeexYMECxo8fX9bG91f3k08+SevWrVm+fDkej4eYmJgqP5Oq8t5779G1a9dy0zMzMyttX/H6SVX9oVd2raWy7TZ2WlxM0bp13iODVRT++itFq1c75/sBIiOJ7tKFhIGnENP9cGIO705M166E2ZGXqYKV2giy/Px8du3aVTY8b948jjzySMD5BT9z5kwAZs6cuc+59JkzZ3L22WfTokULCgoKCAsLIywsjILS/+krePvtt8vejz/+eAByc3Np165d2foqk5ubS0pKCmFhYbz22mu493NnxxlnnMEzzzxTtpP/6aefqmx/+umnM23aNFzei5Pbt2+nW7dupKens3btWgBee+01Tj75ZLp168b69etZt24dAG+++eYBb7ch8hQUULD0J7a//jpb7ruPPy68kDXH9GH9BReSce+97HzfudzX7IILSHno33R+/z26LVnMIR+8T9uHHqLlsMuJ693bkoPZL7t1JcgyMzO54IILAOduocsuu4wzzzwTgHHjxnHJJZfw4osv0rFjR959992y5QoKCpg5cybz5s0D4LbbbuOiiy4iKiqq3A7TV1FREf369cPj8ZS1GT9+PEOGDKFdu3Ycd9xxrF+/3u+yN9xwAxdddBHvvvsup5xyyn7P6f/jH//g1ltvpWfPnqgqqampVd6Get111/Hbb7/Rs2dPIiMjGT58OKNHj+bll19myJAhZRepR44cSXR0NDNmzODss88mKSmJ/v37s3LlygPabkPh2r6dvP+bQ+5HH1G4YkXZBeLw5s2JOfxwEq66kuju3YnpfjhRnTraKSFTI6Sqw/z6pk+fPlrxwuqqVasaxfnp0r4wKt5K2xg1lL+5p6iI3fPnkztrNrsXLQKXi+ju3WkycCAxRxxOTPfuRLRpYw8HmoMiIktUtY+/eXYEYUwdoh4Pe5YuJXfWLPLmfopn1y4ikpNpedWVNDvvfGK6HhbqEE0jYgmigUhPTw91COYgFK1fT+7s2eTN/oiSzZuRuDiaDhpEs/PPI65fPztlZELCEoQxIeLasYO8OXPInT2bwuU/Q1gY8ccfT6tbbqbJaacRFhcX6hBNI2cJwpha5CkuZvf8BeTOns3uL790rit07UryHXfQ9JxziGztrzSZMaFhCcKYIFNV9vz0E7mzZpP3ySd48vKIaNWKlldcQbPzzyOmW7dQh2iMX5YgjAmiwlWryLj3Pgp//RWJjaXJaafR7PzziT/+OLuuYOo8e1AuyNasWUOvXr3KXk2bNuWpp54CnGcU2rVrVzZvzhznwfGvv/6anj17cuyxx5Y9RLZz507OOOOMKp8+Dob09PSyB/sWLFjAOeecU2X7QNpUVzDWGWye4mKyn36a9UMuoSQri5SH/k3aokW0e/wxEvqfYMnB1At2BBFkXbt2ZdmyZYBT0bRdu3ZlD84BjBkzhrFjx5Zbxsp91297Vqwg4557KPp9Lc3OP5/Wd48jvHnzUIdlTLXZEUQt+vzzzzn00EPp1KlTle0OtNz3XXfdRd++fenbt2/ZkcdHH31Ev379OProoznttNPKaiWNHz+eESNGcPrpp3PllVeSnp7OiSeeSO/evenduzfffPNNlTHm5+dzzTXXcOyxx3L00Ucza5a/orx7ud1uxo4dS48ePejZsyfPPPNM2b/J0UcfTY8ePbjmmmsoKioCYO7cuXTr1o3+/fvz/vvvl62nututTZ7CQrImTiR96N9w5+2iw/RptH10giUHU281rp+Nn4yDrStqdp1tesBZEwJq+tZbb5VVXC01ZcoUXn31Vfr06cOkSZNo0aKFlfuuh+W+C5b+RMa991K8fj3Nhwwh+c47CG/SJKQxGXOw7AiilhQXFzN79myGDBlSNm3UqFGsW7eOZcuWkZKSwu233w7sLfc9f/58/vjjj3LlvocNG1ZpxVTfct/ffvst4JT7PuOMM+jRowePP/44v/yyt2p6xXLfw4cPp0ePHgwZMoRff626++958+YxYcIEevXqxYABAw6o3PeaNWv2Kfe9cOFCVq9eXVbuW0QYNmzYAW832DwFBWx9+GH+vPxytKiIji+9SMqD/7LkYBqExnUEEeAv/WD45JNP6N27d7l+HHyHhw8fvs+FWCv3Hfh2QyH/+x/IuO8+SjZupMVll5F8+21WIdU0KHYEUUvefPPNfU4vZWRklA1/8MEHZXcLlbJy33Wz3Ld7dz5b//UvNlx1FYjQ6bVXaXP/Pyw5mAancR1BhEjp+f3p06eXm37nnXeybNkyRITU1NRy863cd90s9737q6/JuP8fuDK20vLqq2l1y82EeU/TGdPQBK3ct4i8BJwDZKnqkd5pbwOl5waaAztVtZefZdOBXYAbcFVWirYiK/dt5b4hOH9zd14emY89Ru5/3yPqkENIeejfxB19dI1uw5hQCFW571eAKcCrpRNUtex2FBGZBPjpIb3MKaq6LWjRGROgXQsWsPWf43FlZ5M4fDhJo28kLDo61GEZE3TB7JN6oYik+psnzhXIS4CBwdp+Y2Plvmuee+dOtj78MHmzPyI6LY32U6YQ2+PI/S9oTAMRqmsQJwKZqvp7JfMVmCciCkxX1RmVrUhERgAjADp27FjjgZrGadeCBWTc9w/cO3eSdMMNJI28HomKCnVYxtSqUCWISwH/V1odJ6jqFhFJBj4TkdWqutBfQ2/ymAHONYiaD9U0Nru//JJNo28iOi2Nji88b9VWTaNV6wlCRCKAC4FjKmujqlu871ki8gHQF/CbIIypSQVLf2LTLbcSc9hhdHx1JuEJCaEOyZiQCcVzEKcBq1V1k7+ZIhIvIk1Kh4HTgZW1GJ9ppAp/+42NI0cS2bo1HZ6fYcnBNHpBSxAi8ibwLdBVRDaJyLXeWX+jwuklEWkrInO8o62Br0RkOfAD8H+qOjdYcdaGyZMnc+SRR3LEEUeUlfoG52GxQYMGkZaWxqBBg9ixYwdg5b5rY50VFW/azMbrhhMWE0OHF18kIjExqNszpj4IWoJQ1UtVNUVVI1W1vaq+6J1+tapOq9B2i6oO9g7/oapHeV9HqOq+NSXqkZUrV/L888/zww8/sHz5cj7++GN+/925Nj9hwgROPfVUfv/9d0499VQmTHBKgZSW+3744YeZOnUqQNDLfTdmrpwcNl57LZ7CQjq88DxR7duFOiRj6gQrtRFkq1at4rjjjiMuLo6IiAhOPvlkPvjgAwBmzZrFVVddBTiF6j788EPAyn1D7ZX7du/ezcbhIyjJzKTDtGnEeAsHGmMaWamNR394lNXbV9foOru17MZdfe+qdP6RRx7JvffeS05ODrGxscyZM4c+fZyHFjMzM0lJSQEgJSWFrKwsACv3XUvlvj1FRWy6cTSFv/1Gh2enENfbnow2xlejShCh0L17d+666y4GDRpEQkICRx111H57byst9w2wcOHCcuW+IyMjmTRpUrlKsKV8y32PGTMGcMp9Dx06lIyMDIqLi+ncuXNZ+4rlvkePHs2yZcsIDw/nt99+qzLGefPmMXv2bCZOnAhwQOW+ly9fvk+572effZYBAwaUlfsGGDZsGDNmzKhyu9UtraFuN1vG3kHB99/T9vHHSKji6MyYxqpRJYiqfukH07XXXsu11zrX6O+55x7at28POOW+MzIySElJISMjg+Tk5HLLWbnvwLdbHarK1vEPsOuzz2h9z900O/fcA16XMQ2ZXYOoBaWnjjZs2MD7779f9kv/vPPOKyvBPXPmTM4///xyy1m57+CU+85+ajI7332XxOuvp+WVV1Z7eWMai0Z1BBEqF110ETk5OURGRvLss8/SokULAMaNG8cll1zCiy++SMeOHXn33XfLlrFy38Ep97195kxypk+n+SWX0OrWWwJezpjGKGjlvkPByn1buW+o/G+eO2sWW+4aR5NBg2j31JNIeHgIojOmbqmq3LedYjKNwq4FC9hyz73E9etH24mPW3IwJgB2iqmBsHLflStYupTNt44hpmtX2j87xfpyMCZAdgRhGrTCNb+xceQoItu0sfpKxlSTJQjTYBVv2sTG665z6iu98ILVVzKmmuwUk2mQXDk5bLj2WjzFxXR67VWrr2TMAbAEYRoc9XjYMHw4rswsOr70ktVXMuYA2SmmWlBZue/x48fTrl07evXqRa9evZgzx6l4buW+D3yd6vHg3r6dot9+p/3Tk62+kjEHwRJEkFVV7htgzJgxLFu2jGXLljF48GDAyn0fKFWlZNMmtLiYto88TMJJJ4U6JGPqNUsQQVZVue/KWLnvAyv37crIwJ2XR3jTplZfyZga0KiuQWx9+GGKVtVsue/o7t1oc889lc6vqtw3wJQpU3j11Vfp06cPkyZNokWLFlbu+wDKfceo4tq+nYjERMJ27tzvv5UxZv+C2eXoSyKSJSIrfaaNF5HNIrLM+xpcybJnisgaEVkrIuOCFWNt8C33feaZZ5Yr9z1q1CjWrVvHsmXLSElJ4fbbbwf2lvueP38+f/zxR7ly38OGDau0Yqpvue9vv/0WcMp9n3HGGfTo0YPHH3+cX375pax9xXLfw4cPp0ePHgwZMoRff/21ys81b948JkyYQK9evRgwYMABlftes2bNPuW+Fy5cyOrVq8vKfYsIw4YNq3K7f6anU7JlCxIVRYSfMujGmAMTzCOIV4ApwKsVpj+pqhMrW0hEwoFngUHAJuBHEZmtqlXvsQJQ1S/9YKqq3Hep4cOH73Mh1sp9B7bdkq1bcW3bRlRqZyTMzpoaU1OC2Sf1QmD7ASzaF1jr7Zu6GHgLOH8/y9RplZX7zsjIKGvzwQcflN0tVMrKfe+/3PeSb7/FtS2H8BYtCE+ofq9yxpjKheIaxGgRuRJYDNyuqjsqzG8HbPQZ3wT0q2xlIjICGAHQsWPHGg61ZlRW7vvOO+9k2bJliAipqalMnz69bBkr9x1Yue+OrZJ5f+pzRNqpJWNqXFDLfYtIKvCxqh7pHW8NbAMUeBBIUdVrKiwzBDhDVa/zjl8B9FXVm/a3PSv33bjKfbu2baNk61aiOnQgvFmzsumN5W9uTE2oM+W+VTVTVd2q6gGexzmdVNEmoIPPeHtgS23EZ+oPT3ExJVlZhDdpQljTpqEOx5gGqVZPMYlIiqqWnni/AFjpp9mPQJqIdAY2A38DLqulEOutxlTuW1Wdu5aAiJSUoDw8aIwJYoIQkTeBAUCSiGwC/gkMEJFeOKeY0oHrvW3bAi+o6mBVdYnIaOBTIBx4SVV/2XcLprFy5+bi2b2byDYphEVFhTocYxqsoCUIVb3Uz+QXK2m7BRjsMz4HmFODsdivzAZCXS5cGVsJi40lPLHlvvMbUBe6xoRag79pPCYmhpycHNtxNBAlW7eiHjeR7dr5fa4iJydnv89wGGMC0+BLbbRv355NmzaRnZ0d6lDMQfIUFeHOySEsIYHwSm7XjYmJKXsQ0RhzcBp8goiMjKRz586hDsMcJM+ePfxx3vlIWBidZ8+yfqWNqQUNPkGYhmHbs89SsnEjHWfOtORgTC1p8NcgTP1X+Ouv5Lz8Cs0uvoj4fv4enTHGBIMlCFOnqctFxj/uJ7x5c1qPHRvqcIxpVOwUk6nTtv/nPxT+8gvtnphEePPmoQ7HmEbFjiBMnVW8aTPZk58m4eSTaXLWWaEOx5hGxxKEqZNUla0PPICI0Oaf99uDjsaEgCUIUyflffx/5C9aRKtbbyWybdtQh2NMo2QJwtQ5rh07yHzkEWKO6kmLy61OozGhYgnC1DlZjz6GOy+PlH89iISHhzocYxotSxCmTsn/5htyP/yQxGuvJabrYaEOx5hGzRKEqTM8e/aQ8c/xRHXqRNINo0IdjjGNnj0HYeqMsnIar7xi5TSMqQPsCMLUCWXlNC66kPjj+oU6HGMMliBMHVCunMYdd4Q6HGOMV9AShIi8JCJZIrLSZ9rjIrJaRH4WkQ9EpHkly6aLyAoRWSYii4MVo6kbSstptLn3HiunYUwdEswjiFeAMytM+ww4UlV7Ar8Bd1ex/Cmq2ktV+wQpPlMHWDkNY+quoCUIVV0IbK8wbZ6quryj3wHW9Vcjl/ngg1ZOw5g6KpTXIK4BPqlkngLzRGSJiIyoaiUiMkJEFovIYutWtH7J/+47dn/5JUk33mDlNIypg0KSIETkXsAFvF5JkxNUtTdwFnCjiJxU2bpUdYaq9lHVPq1atQpCtCYYVJWsSU8QkZJCi2HDQh2OMcaPgBOEiESJSE8R6SEiUQe6QRG5CjgHuFxV1V8bVd3ifc8CPgCsG7EGZten8yhcsYJWN91kzzwYU0cFlCBE5GxgHfA0MAVYKyLVvqIoImcCdwHnqWpBJW3iRaRJ6TBwOrDSX1tTP2lJCdlPPkl0WheanX9eqMMxxlQi0CepJ+HcVbQWQEQOBf6Pyq8hICJvAgOAJBHZBPwT566laOAz7wXJ71R1pIi0BV5Q1cFAa+AD7/wI4A1VnXsAn83UUTvfe5/iP/+k/XPPWTE+Y+qwQBNEVmly8PoDyKpqAVW91M/kFytpuwUY7B3+AzgqwLhMPeMpKCD72SnE9u5NwikDQh2OMaYKgSaIX0RkDvAOzh1GQ4AfReRCAFV9P0jxmQZm+6uv4c7eRvLkyXZbqzF1XKAJIgbIBE72jmcDLYFzcRKGJQizX64dO8h54QUSBg4krnfvUIdjjNmPgBKEqv492IGYhi9n+gw8BQUkj7k11KEYYwIQUIIQkZdxjhTKUdVrajwi0yCVbN7Mjtdfp9lf/0p0WlqowzHGBCDQU0wf+wzHABcAW2o+HNNQZT8zBURoddPoUIdijAlQoKeY3vMd997C+r+gRGQanMLffiN31ixaXn01kSkpoQ7HGBOgAy21kQZ0rMlATMOV/eRThCUkkDhieKhDMcZUQ6DXIHbhXIMQ7/tWnCeijalSwZIl7J4/n1ZjxhDRokWowzHGVEOgp5iaBDsQ0/CoKlkTJxHRqhUtr7wi1OEYY6qpygQhIlXerK6qS2s2HNOQ7J4/nz0//USbBx4gLDY21OEYY6ppf0cQk7zvMUAfYDnOaaaewPdA/+CFZuozdbvJeuIJolJTaX7RhaEOxxhzAKq8SK2qp6jqKcCfQG9vvwvHAEcDa6ta1jRuuR/OonjtOlqNGYNEBHo3tTGmLgn0LqZuqrqidERVVwK9ghKRqfc8hYVkP/MMMT170uT0QaEOxxhzgAL9abdKRF4A/oNzF9MwYFXQojL12o7X38C1dSttH33UCvIZU48FmiD+DowCbvGOLwSmBiUiU6+58/LYNmMG8SeeSHw/6wjQmPos0NtcC0VkGjBHVdcEOSZTj+U8/wKe3FySbxsT6lCMMQcp0C5HzwOWAXO9471EZHYQ4zL1UElmFttfe42m555LTPfuoQ7HGHOQAr1I/U+gL7ATQFWXAalVLSAiL4lIlois9JnWUkQ+E5Hfve9+H60VkTNFZI2IrBWRcQHGaEJs27PPom43rW65OdShGGNqQKAJwqWqudVc9yvAmRWmjQM+V9U04HPveDkiEg48C5wFHA5cKiKHV3PbppYV/bGene+9R4u//Y2o9u1DHY4xpgYEepF6pYhcBoSLSBpwM/BNVQuo6kIRSa0w+XxggHd4JrCAfWs69QXWevumRkTe8i73a4CxmhDIfuopwqKjSRp5fahDMfWIRz241e28e9xl46qKBw+qiqJ4dO9wuXm+86l8vPQdcN6VveOlbXzGS+2zrPqso4KK83zb+F1u31VUuu79zYsMi6RfSr9KlztQgSaIm4B7gSLgDeBT4N8HsL3WqpoBoKoZIpLsp007YKPP+Cag0k8uIiOAEQAdO1qB2VDYs3w5u+bNI2n0aCISE0MdjvFSVQrdhRSUFFDgKvD7XuQuosRTQom7xHkvfVUcr6SNy+MqG3d5XLg8rvI7/QrvpYnArc7L1IzEmEQWDF1Q4+sN9C6mAuBeEXlYVfNrPIry/N04X2lKVdUZwAyAPn36VJ56TVCoKlmTniA8MZGWV18d6nAaFLfHzY6iHeTsyWHbnm3kFOaQsyeHvOI8vzv7Pa49+0yr6tdoZSIkgsjwSCLDvC/f4QrTYiNiy8bDJZzwsHDCJZwwCSt79x0Ol3DCwsKIkIjy08P2XUaQsncRKTcNcMZF9s7HO+6dVrYsgvOfdxjKLVfKd9y3bdmyPvPKvft51qdsWZ/5/tr7tguUv+1FhAWnWkGg5b7/ArwAJAAdReQo4HpVvaGa28sUkRTv0UMKkOWnzSagg894e6z3ujor/6uvKPjhB1rfdx/hCfGhDqfO893p5+zJIafQu/P3HfYmgh2FO/zu4CMkgrjIOOcV4X1FxtE6rjWxkbFl437fvcOxEbHERcYRHR5NVFhU2Q4/IiyibAdsTKBp50ngDGA2gKouF5GTDmB7s4GrgAne91l+2vwIpIlIZ2Az8DfgsgPYlgky9XjImvQEkR060OKSIaEOp84ocZewPm89a3es5fedv7N2x1q25G9h255t7CzaiUc9+ywTEx5DYmwiibGJtE9oz1GtjiIpNonEGGea73BcRJw9oW5qRcDHJaq6scKXssoTiN5uSQcASSKyCedW2QnAOyJyLbABGOJt2xZ4QVUHq6pLREbjXOcIB15S1V8C/0imtuT93/9RtHo1bSdORKKiQh1OrfOoh827N/P7jt9Zu3NtWUJIz03HpS7A+bWf2iyV9k3a0yOph7Ojr7DDT4pNsp2+qZMCTRAbvaeZVESicO5iqrIWk6peWsmsU/203QIM9hmfA8wJMDYTAp7iYrKfmkz04d1pOvisUIcTVKpKTmFOWSIoSwg717LHtaesXbuEdqQ1T2NAhwGkNU+jS4sudG7amcjwyBBGb8yBCzRBjAQm49xhtBnn1/2NwQrK1H0733qbks2b6fDAA0hYwzpnnVecx6JNi/g5++eyhLCjaEfZ/JYxLUlrnsaFaReWJYIuzbsQH2nXYEzDEuhdTNuAy4Mci6kn3Lt3s23qVOKOO474E/4S6nBqRGZ+JvM3zueLDV/w49YfcamL2IhY0pqnMbDjQLo070JaizS6NO9CYqzdymsah0DvYjoE5wjiOJxbTr8FxpQ+zGYal+0vvYx7xw6Sb7+tXp83X5+7ns83fM78DfP5edvPAKQ2TeXKI65kYMeB9EjqYXf0mEYt0FNMb+CUv7jAO/434E2qeIDNNEwlmVnkvPIKTc48k9gePUIdTrWoKr/k/MLnGz7niw1f8Eeu8/vmiMQjuPnomzm146l0bta5Xic9Y2pSoAlCVPU1n/H/eO80Mo1M9lNPQUkJybffFupQAlLiKWFJ5hK+2PAFX2z4gsyCTMIlnD6t+zC061AGdhxIm/g2oQ7TmDop0AQx31tV9S2cU0xDgf8TkZYAqro9SPGZOmTPyl/I/eADEq+7lqgOHfa/QIjsce3hm83f8MXGL1iwcQF5xXnEhMfwl7Z/4ebeN3NSu5NoHtM81GEaU+cFmiCGet+vZ2/ZCwGu8Y4fUsNxmTpGVcmc8AjhLVuSOHJkqMPZh0c9fL7hcz5e9zHfbPmGQnchTaOaMqDDAAZ2GMhf2v2F2IjYUIdpTL0SaIK4C5irqnki8g+gN/Cgqi4NXmimLtn16Tz2LF5CmwceIDwhIdThlFFVFm1exDM/PcPq7atpHdeaC9IuYGDHgRzT+hgiw+wZBGMOVKAJ4j5VfUdE+gODgEk4fVLbRepGwFNURNbEiUQfdhjNL74o1OGUWZq5lMlLJ7M0ayntEtrxcP+HGdx5MOFh4aEOzZgGIdAEUVpW42xgmqrOEpHxwQnJ1DXbX32Vkk2b6PjyS0h46He+q7evZvLSyXy1+Staxbbivn73cWHahfbEsjE1LNAEsVlEpgOnAY+KSDSB90Zn6jHXtm3kTJtOwimnEH/88SGNJT03nWeXPcvc9Lk0jWrKmGPGcGm3S+3agjFBEmiCuASn+9CJqrrTW6r7juCFZeqK7MlP4ykqIvnO0P25t+ZvZdryaXy49kOiwqMY3mM4Vx95NU2jmoYsJmMag+p0GPS+z3gGkBGsoEzdULhmDTvfe4+WVwwjunPnWt/+jsIdvLDiBd5a/RYePAztOpThPYeTFJtU67EY0xgFpxsiU+85t7VOILxJE5JuqG6/UAdnd/FuXvv1NWb+OpM9rj2ce8i5jOo1inYJ7Wo1DmMaO0sQxq/d8+dT8O13Tk9xzZrVyjaL3EW8tfotXljxAjuLdnJax9MYffRoDm1+aK1s3xhTniUIsw8tLibr0ceIOvRQWgy9JOjbc3lcfLj2Q6Ytn0ZmQSbHpxzPzb1v5sikI4O+bWNM5SxBmH1sf+MNiv/8kw4zpiORwb119IsNX/DEkif4M+9Peib15OH+D9M3pW9Qt2mMCUytJwgR6Qq87TPpEOB+VX3Kp80AnP6q13snva+q/6qlEBs1144dbHtuKvH9+5Nw0oF0Ox6YEncJExdP5I3Vb9CleRcmnzKZUzqcYpVUjalDaj1BqOoaoBeAiITj9FD3gZ+mi1T1nFoMzQDbnpmCJz+f1nfdGbRtZOzOYOyXY/l5289ccfgVjDlmjJXEMKYOCvUpplOBdar6Z4jjMEDR2rXsePttWgy9hOi0tKBs4+vNXzNu0ThKPCVMOnkSp6eeHpTtGGMOXqifhi7teMif40VkuYh8IiJHVLYCERkhIotFZHF2dnZwomwkMh99jLC4OJJuuqnG1+32uHlu2XOM+t8okmKTeOvstyw5GFPHhSxBiEgUcB7wrp/ZS4FOqnoU8AzwYWXrUdUZqtpHVfu0atUqKLE2BrsXLiR/0SKSbriBiBYtanTdOwp3cMPnNzB1+VTOOeQcXh/8OqnNUmt0G8aYmhfKU0xnAUtVNbPiDFXN8xmeIyLPiUiSqm6r1QgbCS0pIfPRx4js1JGWl19Wo+v+Oftnbv/ydnL25HD/8fdzcdrFdiHamHoilAniUio5vSQibYBMVVUR6YtzpJNTm8E1JjveeYfideto/+wUJCqqRtapqryx+g0mLp5I67jWvDb4NY5IrPRMoTGmDgpJghCROJx+Ja73mTYSQFWnARcDo0TEBewB/qaq6m9d5uC4c3PZ9vQzxB13HAkDB9bIOgtKChj/zXg+Sf+Ek9ufzEP9H6JZdO08jW2MqTkhSRDe4n+JFaZN8xmeAkyp7bgao23PTcWdl0frcXfVyKmfdTvXMWbBGP7M+5Nbet/CNUdeQ5iE+l4IY8yBCPVtriaEitavZ/vrr9P84ouJ6dbtoNc35485jP92PLERsTw/6Hl7ItqYes4SRCOW9fhEwqKjaXXLzQe1nmJ3MY/9+Bhvr3mb3sm9efzkx0mOS66hKI0xoWIJopHK//Zbdn/xBa1uu42IpAPvX2HL7i2M/XIsK7at4KrDr+KWY26xp6KNaSAsQTRC6naTOeFRItu1o+VVVx7wer7a/BXjFo3D7XHz5IAnOa3TaTUYpTEm1CxBNEI733uPojVraPfUk4RFR1d7ebfHzbSfpzF9+XTSWqTxxIAn6NS0UxAiNcaEkiWIRsa9ezfZk58m9phjaHLGGdVevqCkgNu+vI2vN3/N+Yeez73H3UtsRGwQIjXGhJoliEYmZ/p03Dk5tJ42rdq3teYW5XLD/27gl5xfuP/4+xly2JAgRWmMqQssQTQixRs3sv2VmTT761+J7VG93toy8zMZ+b+RbMjbwBMDnmBgx5p5qM4YU3dZgmhEsiZOgogIWo0ZU63lNuRtYMRnI9hRuIOpp0215xuMaSQsQTQSBT/+yK5PPyXp5puIbB34Mwprtq/h+s+ux61uXjrjJY5IsnpKxjQWVgOhEfAUF7P14UeIaNOGxL//PeDlfsr6ib/P/TsRYRHMPHOmJQdjGhk7gmgEsiZOpGjVKto/O4Ww2MDuOFq0aRG3LbiNNvFtmDFoBikJKUGO0hhT19gRRAO363//Y8err9HiiitocuqpAS0z54853PzFzXRu1plXznzFkoMxjZQliAaseNNmttxzLzFHHEHyHWMDWuat1W8xbtE4eiX34qUzXiIxNnH/CxljGiQ7xdRAaXExm2+7DTwe54np/XQEpKrM+HkGU5ZNYUD7ATx+8uPERMTUUrTGmLrIEkQDlfXEkxT+/DPtJk8mqkOHKtt61MPjPz7Of1b9h3MPOZcHTnjACu4ZYyxBNES7vpjP9ldeocVll9H0jNOrbOvyuPjnN/9k9rrZXN79cu489k7r4McYA4Suy9F0YBfgBlyq2qfCfAEmA4OBAuBqVV1a23HWRyVbtrDl7ruJPrw7yXfdWWXbIncRd3x5B/M3zueGXjcwsufIGulVzhjTMITyCOIUVd1WybyzgDTvqx8w1ftuqqAlJWwecxu4XLR/supKrbuLd3Pz/Jv5ceuP3N33bi7rflktRmqMqQ/q6imm84FXVVWB70SkuYikqGpGqAOry7Keeoo9y5fT7olJRHWqvPz29sLtjPrfKH7b/hsTTpzA2YecXYtRGmPqi1CdbFZgnogsEZERfua3Azb6jG/yTtuHiIwQkcUisjg7OzsIodYPuxYsYPuLL9F86FCaDh5cabut+Vu56pOrWLdzHZMHTrbkYIypVKiOIE5Q1S0ikgx8JiKrVXWhz3x/J8LV34pUdQYwA6BPnz5+2zR0JVu3kjHubqK7daP13eMqbbc+dz0jPhvB7uLdTB80nWNaH1OLURpj6puQHEGo6hbvexbwAVCxPOgmwPfezPbAltqJrn5Rl4vNt92OFhfT7sknCIvx/+zCrzm/ctUnV1HsLualM16y5GCM2a9aTxAiEi8iTUqHgdOBlRWazQauFMdxQK5df/Ave/LT7Fm6lDYPPEB0585+23yz5Rv+PvfvxEbE8upZr9I9sXstR2mMqY9CcYqpNfCB93bKCOANVZ0rIiMBVHUaMAfnFte1OLe5Bl6CtBHZvWgROc8/T/MhF9Ps3HP8tvlo3Ufc//X9dG7emamnTqV1fOtajtIYU1+Jc6NQw9CnTx9dvHhxqMOoFSWZmaz/6wVEJCWR+s7b+1RpVVVe/uVlnlzyJH3b9OWpU56iSVSTEEVrjKmrRGRJxWfRStXV21xNFdTlYsvtY/EUFjp1liokB7fHzWM/PsYbq9/grNSz+Hf/fxMVXnUtJmOMqcgSRD2U/eyzFCxeTMqER4g+9NBy84rcRdy96G4++/Mzrjz8Sm7vc7uVzjDGHBBLEPXM7q+/JmfadJpdeCHN//rXcvNyi3K5+YubWZq1lDv63MGVR1wZmiCNMQ2CJYh6pCQriy133EnUoYfQ5r57y83L2J3BqP+NYsOuDTx+0uOc2fnMEEVpjGkoLEHUE+p2s+WOO/EUFNBp5iuExcWVzVuzfQ03/O8GClwFTDttGn1TKj5WYg6aKnjc4C4CdzG4S8DlM+wu8r4Xe6d7h8teJaBuZx3qAY/LO+yd5jusbp/5Hu98V/nlVZ13vO++08qmq59pHj/L697P6Ax4hzXwaRWXLzfuO63CuL/l9lm2wrxK21SnXQDL+W1WR2/qiUuE6z6r8dVagqgntj03lYLvvyfloYeITksrm/5Dxg/cMv8W4iLjmHnWTA5rcVgIo6wD3CVQtAuK831ePuPl5u32vvzNywfXHnD57OQD3YkcrLAIkHAIC9/77jssYc6w4B0OA2TvsEiFabJ3WsXpyL7vCIR5r1uVTcNPu4rTqDDdZ9zfNH+Vg8tNkyrmVdKmOu0CWs5vwwDb1aKYpkFZrSWIeiD/u+/Y9txzNDv/PJpdeEHZ9Lnr53LPV/fQsUlHpg2aRpv4NiGMMkhUoTAX8rNhdxbkZ8HubO97ljO9bF42lBQEvu7IeIgqfSVAdALEtYTmHZzxyFgIj4LwSAiP3jscEe2dFrXvK8LPtPBI51W2s49wdtBlO/0In2G7ocDUHZYg6jhXdjabx95BVOfOtLn//rL+Gmb+MpOJiyfSO7k3Tw98mmbRzUIc6QHIz4Ftv0He5r07//xsnwTgfXcX77ushDmH1fHJkNAKOvSDhGSIbe7s3Et3+mXD8RDdZO9wZJyzUzbGVMoSRB2mbjeb77wTz65ddHzxRcLi4/Goh4mLJ/Lar68xqNMgHjnxEaLDK+/3IeQ8HsjdANm/OcnA91WQU75tWATEt3JeCcnQqruz849PdsZLp8cnO7/0bQdvTFBZgqjDtk2dRsG339HmwX8R0/Uwit3F3PvVvcxNn8tl3S7jzmPvJLyu7CRL9kDOWu/O/3fIXuO85/wOrsK97eISIakrdDsHWnWFpMOgWQdnxx/T3E6xGFOHWIKog9TjIXvy0+RMn07Tc8+l+cUXs6t4F7fMv4Uft/7IbcfcxtVHXB2a7kHdLti6HLau3HskkL0Gdm5g70VcgRadnJ3/ISc776Wv+MTaj9kYc0AsQdQxnqIiMu6+m7w5n9B8yMW0uf9+sgqyGPX5KNbnrueREx/hnEP8F+YLitKEkP6V8/rzW+euIICIGEhMg3bHQK/LICnNOTpIPNS5wGuMqdcsQdQhrh072HTDjez56Sda3X4bidddx7qd6xj1+Sh2Fe/iuVOf4/i2xwc3CI8btv4M6xc5CWHDt1CU58xLOgx6XgKp/aFdb2jW0U4JGdOAWYKoI4rWr2fj9SNxbd1Ku6eepOmZZ7Ikcwk3fXET0eHRvHLmK3Rr2a3mN+xxw9YVkO5NCH9+szchJKbBkRdB5xOhU39oYqXCjWlMLEHUAQU//sjG0TchYWF0nPkKkUcdydTlU5mxfAbtm7Rn2qBptEvw2yV39XnckLnSSQbrF3kTQq4zL7ELHHkhpJ7oHCU0aYDPVRhjAmYJIsRyP/qIjHvuJbJ9ezpMn8bmZm7umXMlK3NWMrjzYO7pd8/BP+OwKxNWzYZ18+HPr5wHzwBaHgJH/NWbEE6Apm0P+vMYYxoOSxAhoqpse+45tj0zhbhjj6Xt00/xztZPeGrhU0RHRPP4yY9zZupBFNzLz4FVs2Dl+87RAgotUqH7eXuPEJrV0FGJMaZBqvUEISIdgFeBNoAHmKGqkyu0GQDMAtZ7J72vqv+qxTCDSouLyfjH/eTOmkWz889H7r6RG364i+8yvqN/u/488JcHSI5Lrv6K9+yE1R87SeGPBU5xt8Q0OPlOOOJCSA7CNQxjTIMViiMIF3C7qi4VkSbAEhH5TFV/rdBukarW4v2ctcOdm8umm26m4IcfSBo9mu/O7MAjc4biUhf3H38/F6ddXL3nG4p2wZpPnKSw9n/gKYHmneCEm52k0KZHNYqQGWPMXrWeIFQ1A8jwDu8SkVVAO6BigmhwijduZOOI6ynZtIlmD93PI4k/8tnX0zg6+WgeOuEhOjTtEOCKCuD3T52k8Ps850nlpu2g3/XORea2vS0pGGMOWkivQYhIKnA08L2f2ceLyHJgCzBWVX+pzdhqWsFPP7HphhvB42H7hJsZuXs6uRtzubX3rVx9xNX7L5nhKnKOEFa+7xwxlOQ7NYl6X+ncitq+rz2TYIypUSFLECKSALwH3KqqeRVmLwU6qepuERkMfAik4YeIjABGAHTs2DF4AR+EvLlz2XLnXYS3TmbWqJ7M3DaZtBZpTB80na4tu1a+oLvEuZaw8n3n2kJRHsS2hJ5DnKTQ6QQrWGeMCRrREPSQJCKRwMfAp6r6RADt04E+qrqtqnZ9+vTRxYsX10yQNUBVyXnhBbInPYG7x2H889wCftdM/n7k37mx141EhUf5XzDjZ1j2Bqx4x6l4Gt0Mup8LR14AnU92+hYwxpgaICJLVLWPv3mhuItJgBeBVZUlBxFpA2SqqopIXyAMyPHXtq7SkhK2/utf7Hz3v2w57lDuPPEPkuPb80r/V+jduve+C+zOhhXvOokhc4XT0Uy3s6HnUDh0oNNJjTHG1KJQnGI6AbgCWCEiy7zT7gE6AqjqNOBiYJSIuIA9wN80FIc6B8i9axebb7mV/G++Yf7ARKb1Tefirpcwts9Y4iL39iWNq9i5yLzsDeeis8flFL47e5JzB1Jcy9B9CFMnVfzfoOL/Ffv05Fyx/T7zK9lOJd2r1tb/hfXn//a6Izaq5k83h+Iupq/YT6euqjoFmFI7EdWsks2b2XD9SArX/8Hz50SwvG8kU/7yHCe1P2lvo4qnkBJaw/E3wlGX1etnFVSVYreHIpeHohIPRS532XCx20NRiXfc5Z1X4gyXuD24PIrb46HErbg9Wjbu8ihutzPu8niceeXaKCVuT9m4R70vD3hUUQV36TR1YvSdXzq9tK2/5ZXS99Idl+/43vV6Z5VNL23vuzzeNt61eP/dyk0u26nvHQ/mX800BEkJ0Sy+77QaX689SV1D3Lt2sf2119j28ssUluzhsUug7cmn88Fx/6B5TPPKTyH1uhwOOQXCa/9P4fEo+cUudhU6r91FJeSVDhe62FVYwu4iZzyvsMQ7zcWuohL2FPvs7H12/DUpIkwID5O97+FhRJSOhwsRYWHl53vfw8R5iUB4mBDpMy1M8M5zhkvbi3d6xfkiIDjznTuHvcNQtowz7PzmKde+dNy7fu/SZe2ccSqM+59fOqFie99l8DOv3Doqm1/NW6Kral4xloNhd2oHLi4IRw9gCeKgufPy2D7zVbbNfBl2F7A4LYwPTk/g2rPvZ3CH05C1n9XaKaRil4fs3UVk5RWStauIrF1FZOcVeqcVsXNPibPTL00Cxa79/joNE0iIjqBJTCRNYiJIiI6gVUI0cVERREeEER0ZRnREuDMcEUZ0pM9wRLh3vk+bcu3DiYoIIyrC/44+JB0iGWPKWII4QO6dO8mZOZPsma8QVlDI94cJ/3dSPH1OGsLUVv1ovXou/PemGjmFlF/kcnb4Pjt7JwEUkr2rdLyQHQUl+ywrAonx0bRqEk3L+EiSEuLLdvZNfHf8Mc5wQnQETUuHYyKIjwq3HbUxjZQliGpy7dhB1ksvsv0//yF8TxE/dBUWnJrMgH5n82J+MU0Xz4Ksh51TSF0Hw9HD9nsKyeNRMncVsn5bPuu35ZO+LZ/12wpIz8knY+ce8ovd+ywTGS4kN4mhVZNoOiXG0Se1BclNYkhuGk1yk+iy4cT4KCLC7QE6Y0z1WYIIkGv7djbPeI68t94hrLCE77sLP53ekbM6dOKFP5cR+clDgEDH42DwROdBNp9TSKrKtt3FpOfkV0gE+aTn5FNYsvf8fXREGKmJ8RzaKp4T05KcnX2TaO/O3xluHhdpv+yNMUFlCWI/XDk5rHvuCYr+O5vwIhffdxc2DGzDueE7uSbzOyTrB+eJ5mOHQ7dzyI1M4o/s3aSvyWd9djbrcwpI9yaDXUWusvVGhAkdE+PonBjPCV2S6JwUT+ekeFKT4klpGkNYmO38jTGhZQmiEiVZWfzyzMOEffgZ4S4PPxwexu4+4ZzvyaDz9q1o6olsO3kEy+L7s3xHFKtX5bHqi1Vs3rmnbB1hAu1axNI5KYHeHZuT6k0CnZPiadc81k79GGPqNEsQFezZuoWfnrifhDnfEOlWfjhcCO+RzwXhe9iTcCzfx17GP/b0YsnaMPascgMbCA8TDkmK55hOLbj8uI6kJTehc1I8HVrGEh1htZKMMfWTJQiv7X+u4adHx5K0cC1N3bDscKXp4fl0juzCx0XnMLHwGPLyE2geF0n3Nk25tG9TuqU04fCUpnRJTiAm0hKBMaZhafQJYkdmOp/dchFdVxSQrLCqu4fcw1ryS9jJrG9+Eh1S2tA9pSmTU5rSPaUprZtG28VhY0yj0OgTRNOWbUnaVMCa7pHs6j+IFv2G069DG65o3cSOCowxjVqjTxDhkVGc+NliImPjQx2KMcbUKXYbDVhyMMYYPyxBGGOM8csShDHGGL8sQRhjjPHLEoQxxhi/LEEYY4zxyxKEMcYYvyxBGGOM8Uu0AfWILiLZwJ+hjgNIAraFOgg/LK7qsbiqx+KqnroSVydVbeVvRoNKEHWFiCxW1T6hjqMii6t6LK7qsbiqp67G5ctOMRljjPHLEoQxxhi/LEEEx4xQB1AJi6t6LK7qsbiqp67GVcauQRhjjPHLjiCMMcb4ZQnCGGOMX5YgjDHG+GUJIkhE5BAReVFE/hvqWHyJSHcRmSYi/xWRUaGOp5SIDBCRRd7YBoQ6nlIicqI3phdE5JtQx1NKRA4XkXdEZKqIXFwH4in3fa8r338/cdWJ77+fuOrk998SxEESkQ4iMl9EVonILyJyC4Cq/qGq19bBuFap6kjgEqDWH9KpLC5Agd1ADLCprsSlqou8/14fAzPrSlzAWcAzqjoKuDLU8VT8vtf2978acdXq978a+4eQfv8rpar2OogXkAL09g43AX4DDveZ/9+6FhdwHvANcFldiQsI805rDbxeV+Lymf8O0LSuxAUkA88CjwNfhzoen/n/rdC+Vr7/1YmrNr//gcYV6u9/ZS87gjhIqpqhqku9w7uAVUC70EZVdVyqOltV/wJcXlfiUlWPt8kOILquxAUgIh2BXFXNqytxqWqWqt4IjKMW6/nUx++7n7a19v0PNK5Qf/8rExHqABoSEUkFjga+F5FE4CHgaBG5W1UfqSNxDQAuxPkSzglVTLBPXBcCZwDNgSkhDKtcXN5J1wIvhywgrwr/XqnAPUA8zlFEqOMp933HeQgsJN///cT1LSH6/u8nrjXUke9/OaE+hGkoLyABWAJcGOpYLC6Lq7HFY3EF52WnmGqAiEQC7+GcO3w/1PGUsriqx+Kqn/GUsrhqnpXaOEgiIjh3t2xX1VtDHE4Zi6t6LK7A1LV4SllcwWEJ4iCJSH9gEbACKL3QdI+qhvr8vsVVDRZX/YynlMUVHJYgjDHG+GXXIIwxxvhlCcIYY4xfliCMMcb4ZQnCGGOMX5YgjDHG+GUJwhhjjF+WIEyjISILRKQ2Sjzf7C3v/Hqwt2VMMFmxPmMCICIRquoKsPkNwFmquv4AtiM4zyd59tu4FtXVuExw2RGEqVNEJNX76/t5bwcr80Qk1juv7AhARJJEJN07fLWIfCgiH4nIehEZLSK3ichPIvKdiLT02cQwEflGRFaKSF/v8vEi8pKI/Ohd5nyf9b4rIh8B8/zEept3PStF5FbvtGnAIcBsERlTof3VIjJLROaKyBoR+WeFz/wcsBToICKPe9e7QkSG+qzjTu+05SIywTvtUO86l4jTK1k37/Qh3nUsF5GF3mlHiMgPIrJMRH4WkbQqPou/uF7xiavc5zMNUKirBdrLXr4vIBVwAb284+8Aw7zDC4A+3uEkIN07fDWwFqdDllZALjDSO+9J4Faf5Z/3Dp8ErPQOP+yzjeY4nbrEe9e7CWjpJ85jcMonxONU6vwFONo7Lx1I8rPM1UAGkAjEAitxejVLxSnDcJy33UXAZ0A4TgcyG3A6njkLp6ObOG+7lt73z4E073A/4Avv8AqcviMAmnvfnwEu9w5HeePw+1n8xHUM8JnP52ke6u+LvYL7siMIUxetV9Vl3uElODuq/ZmvqrtUNRsnQXzknb6iwvJvAqjqQqCpiDQHTgfGicgynCQSA3T0tv9MVbf72V5/4ANVzVfV3cD7wIkBxPmZquao6h7vMv290/9U1e981v2mqrpVNRP4EjgWOA14WVULvJ9hu4gkAH8B3vXGPx0nmQB8DbwiIsNxkg04/SHcIyJ3AZ28cVT1WXzj+gM4RESeEZEzgVrvQMnULrsGYeqiIp9hN86vXHCOLEp/1MRUsYzHZ9xD+e95xeJjCghwkaqu8Z0hIv2A/EpilMqC3w9/26fCdipbt/hZPgzYqaq99tmQ6kjvZzgbWCYivVT1DRH53jvtUxG5rortlYtLVXeIyFE4HdvciNOv8zVVLGvqOTuCMPVJOs5pDoCLD3AdQ6GsymauquYCnwI3eS/EIiJHB7CehcBfRSROROKBC3Cqdu7PIBFp6b2u8lecX/n+1j1URMJFpBXO6bAfcK6DXCMicd44W6rTDep6ERninSbenTgicqiqfq+q9+N0S9pBRA4B/lDVp4HZQM9AP4uIJOH0nfwe8A+gdwCf19RjdgRh6pOJwDsicgXwxQGuY4eIfAM0Ze+v3weBp4CfvUkiHTinqpWo6lIReQVnxw3wgqr+FMD2vwJeA7oAb6jqYnG6ovT1AXA8sBzniOFOVd0KzBWRXsBiESnG6TLzHpy+laeKyH1AJPCWd9nHvRehBec6xXKcPqyHiUgJsBX4l/dU1T6fxU9c7YCXRaT0h+XdAXxeU49ZuW9jaomIXI1zkX10qGMxJhB2iskYY4xfdgRhjDHGLzuCMMYY45clCGOMMX5ZgjDGGOOXJQhjjDF+WYIwxhjjlyUIY4wxfv0/dCvMFtSix3IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "nproc=np.logspace(0,16,num=17,base=2,dtype=int)\n",
    "\n",
    "# p is the parallelizable part of the code, that can be sped up.\n",
    "# n is the number of processors applied to the problem.\n",
    "def t(p,n):\n",
    "    return (1-p)+(p)/n\n",
    "\n",
    "A = {}\n",
    "for p in [ 0.5, 0.75, 0.9, 0.95 ]:\n",
    "    i=\"%d\" %p\n",
    "    A[i] =np.vectorize( lambda n: 1/t(p,n) )(nproc)\n",
    "    plt.plot(nproc,A[i], label=\"%02d%% parallel code\" %int(p*100))\n",
    "\n",
    "plt.xscale('log',base=2)\n",
    "plt.xlabel(\"number of processors\")\n",
    "plt.ylabel(\"speedup\")\n",
    "plt.title(\"Amdahl's law\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "338525a3",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "Large Numbers of Processors Need High Parallelism\n",
    "---------------------------------------------------------------------------\n",
    "\n",
    "In the previous picture, we could see that:\n",
    "\n",
    "* we could never get more than a 20 fold speedup if 5% of the code is serial. \n",
    "* To use higher number of processors, we need higher and higher levels of parallelization. \n",
    "* There limit is, of course, 100% parallel code, usually referred to as \"embarrassingly parallel.\"  \n",
    "* to use larger numbers of processors, the ideal is to write embarassingly (aka Perfectly) parallel code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0d21732c",
   "metadata": {
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEaCAYAAADg2nttAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABG+ElEQVR4nO3dd3xV9fnA8c9zb/YCEpIACXuDDAEtahXRKjhBKa7aatWqVau2LtRaf7ZSceDGvajWgRtHrVRFWxcyZQ+ZYSQhhJCd3Huf3x/nJFxCQkJIcjOe9+t1X2ef89xwOc8933Pu8xVVxRhjjDkQT6gDMMYY0/xZsjDGGFMrSxbGGGNqZcnCGGNMrSxZGGOMqZUlC2OMMbWyZGGaLRF5SUTuruO6c0XkshqW9RARFZGwQ4jleBHJqO/2DUlE/iUiFx1geZ3/bsbUlSULc8jcE3WuiESGOpa6cOM9PtRx1JeqnqKqMwFE5GIR+V+oYxKRSBF5SES2uZ+FJ0QkPGj5QBH5XETyRGSdiJwVtKyriHwnIrtEZHqV/X4iIqOa8r2Y6lmyMIdERHoAxwIKnBnaaExDE5HE4JP+AUwBRgGHAf2AEcCf3X2EAe8DHwKJwOXAKyLSz932VmAm0BOYWJEcRORcYL2qzm+4d2Tqy5KFOVS/Ab4DXgL2aRpxm0OecJtNCkTkaxHpJCIPu98+V4nI4UHrHy4iC0UkX0TeAKKClnUQkQ9FJNvd9kMRSa8SS3f3GPki8qmIdKwteBE5UkTmi8geEckUkQfr8qZFZIqI/OQea0WVb8qbRGSkO36h2wQ2yJ2+TETeq2Z/PUVkt4h43OnnRCQraPkrInK9Oz7X3c9A4CngKPfvuztolx1E5CM3vu9FpHcN76Oiie5y96pgu4jcELTKSUCGiEwXkcMO8Cc5A3hUVXepajbwKHCJu2wA0AV4SFX9qvo58DXwa3d5T+BzVc0DfgB6iUgCTgK67QDHNE3IkoU5VL8B/um+xolIapXl5+B8w+wIlALfAgvd6beABwFEJAJ4D3gZ59vnm8CkoP14gBeB7kA3oBh4vMqxLgB+C6QAEcCN1QWsqser6lx38hHgEVVNAHoDs+r4vn/CuaJqB9yF8025s7vsS+B4d/w4YD0wJmj6y2pi2gDsASqS57FAgZsQqt1OVVcCVwLfqmqcqrYPWny+G1cHYB0wtZb3MxboC5wMTBGRX7jHeAM4EQgAn4rIDyJylYh0qLK9uK/g6XQRaVdlfvDyiuSzDDhJRNrjXJ2sAP4GPKyqu2uJ2zQRSxam3kTk5zgn71mqugDnBHpBldXeVdUFqloCvAuUqOo/VNUPvMHek+NoIBznBFGuqm/hfMsEQFVzVPVtVS1S1Xyck9+YfQ/Fi6q6RlWLcU76w+vwNsqBPiLSUVULVPW7urx3VX1TVbepasA9oa4FjnQXfxkU27HAPUHTY6gmWQRvJyKd3Om33OmeQAKwpC6xud5R1Xmq6sNJ5MNrWf8uVS1U1aU4Sfn8igWqukxVbwK6AnfiJMINIvK6ewUA8C/gOhFJduO/1p0fA6wCsoCbRCRcRE7G+TvEuOvcg/N3+hKYgfM5GAp8ICKvishXInLNQbx30wgsWZhDcRHwqarudKdfpUpTFJAZNF5czXScO94F2Kr7VrbcVDEiIjEi8rTbxLMH+ApoLyLeoPV3BI0XBe37QC7FaWNf5X5rPr0O2yAivxGRxW7T0W6cb8kVzV5fAse6J00vTlI8xr2/0w5YXMNuK65IjsN5f3NxTqpjgP+qaqAusbkO9m+xJWh8E86/xz7cBL8MJ2ntwnnPFfczpgKLcN7bNzhXieVAlqqWAxOB09y4bsBJ5hnufnep6rmqOgznSu8x4A84zVDLgF8AV1Y05ZnQsGRh6kVEonGamMaIyA4R2QH8ERgmIsPqscvtQJqIBDdZdAsavwHoD/zMbTI6riKUehyrkqquVdXzcZqu7gXeEpHYA20jIt2BZ4FrgCS3+WdZRSyqug7nBH0t8JV7JbQD58bu/w5w0v8S5xv28e74/4BjOPDVSEOVje4aNN4N2FYxISJx4jx19TlOE2IacK6qHqaqOQCqWqyq16hqmqr2AnKABW6CQVV/VNUxqpqkquOAXsC8auK4HPhOVZcBQ4D5qloGLGVvs5UJAUsWpr4mAn5gEE4Tx3BgIPBfnPsYB+tbwAdcKyJhInI2e5t1AOJxrkR2i0giTnPIIXNvQCe7J/Dd7mx/LZvF4pyks919/Jb9T2Rf4iSTipP83CrT+1HVtTjv8UKcJLMH50ps0gG2y8S5NxBRS8y1ucO9ehuMc9/nDQARGY+TOM4FngbSVPUqVf0heGMRSRORLuIYDdxB0L+RiAwVkSj3GDcCnXEeigjeRwpwNfB/7qwNwFgRicO5l7H+EN+jOQSWLEx9XYRzj2Czqu6oeOHcdP6VHOQP4Nxvj2cDFwO5OCend4JWeRiIBnbiPH31ySG/A8d4YLmIFOA0gZzn3l85UKwrgOk4CS4T5xvw11VW+xInwX1Vw3RNvgRyVHVz0LTgNPFU53NgObBDRHbWsE5dfIlzI/wz4AFV/dSdvxoY4P624w1VLa1h+944zU+FOI/BTgnaBzhPPm3HuXdxInBSNft6APirqha40/cAJ+A0kc22R2hDS6zzI2PaLvc+ygYg3L0Zbky17MrCGGNMrSxZGGOMqZU1QxljjKmVXVkYY4yplSULY4wxtap3ff/mrmPHjtqjR49Qh2GMMS3KggULdqpqctX5rTZZ9OjRg/nz7bFsY4w5GCKyqbr51gxljDGmVo2WLETkBRHJEpFlQfMSRWSOiKx1hx2Clt0qTg9aq0VkXND8kSKy1F32aJXaQcYYY5pAY15ZvIRTSiHYFOAzVe2LU1ZgCoBbTfI8YLC7zRNB1USfxCku1td9Vd2nMcaYRtZo9yxU9Su3lECwCeztFGYmTnG1W9z5r7u1YjaIyDrgSBHZCCSo6rcAIvIPnAJ2/6pPTOXl5WRkZFBScsDSP6aRRUVFkZ6eTnh4XXrrNMY0B019gztVVbcDqOp2t8okOCWPgzudyXDnlbvjVedXS0Qux7kKoVu3bvstz8jIID4+nh49emCtWaGhquTk5JCRkUHPnj1DHY4xpo6ayw3u6s7ceoD51VLVZ1R1lKqOSk7e78kvSkpKSEpKskQRQiJCUlKSXd0Z08I0dbLIrOin2B1WdEifwb6dr6Tj1NDPcMerzq83SxShZ/8GxjS8QGEhxcuXs+ffn9a+cj00dbKYzd5uNy8C3g+af56IRLr9DfcF5rlNVvkiMtp9Cuo3Qdu0SI888giHHXYYgwcP5uGHH66cv2TJEo466iiGDBnCGWecwZ49e6rd/pNPPqF///706dOHadOmNVHUe/Xo0YOdO51uE+Liau+1tC7rGGPqRgMByrdupeB/X7PrHy+z469/ZdNvf8vaMcezeuQoNk76JVuvuw5/QUHtOztIjXbPQkRew7mZ3VFEMnB6zZoGzBKRS4HNwGQAVV0uIrOAFTi9pV1d0R0j8HucJ6uicW5s1+vmdnOwbNkynn32WebNm0dERATjx4/ntNNOo2/fvlx22WU88MADjBkzhhdeeIH777+fv/3tb/ts7/f7ufrqq5kzZw7p6ekcccQRnHnmmQwa1HBdE/v9frxeb+0rGmMaTaCwkNINGynbsIGyDRso3bCesg0bKdu4EQ1qwvXExxPRsyexo0cT0bMnEb16EtmzJ57o6AaPqTGfhjq/hkUn1rD+VJxO36vOn08r6Xt35cqVjB49mpiYGADGjBnDu+++y80338zq1as57jinW+mTTjqJcePG7Zcs5s2bR58+fejVqxcA5513Hu+///5+yeLiiy8mKiqK5cuXk5mZyYMPPsjpp5/Oxo0b+fWvf01hYSEAjz/+OEcffTRz587lrrvuonPnzixevJgVK1YwceJEtmzZQklJCddddx2XX375Ad/b/fffz6xZsygtLeWss87irrvuapC/mTGtmapSunIlRQsWUrZhPaXrneTgy8zcu5LHQ3h6OhE9exD7s58R0asXET17ENmrF94mvAfbast91OauD5azYlv1TT31NahLAneeMbjG5Ycddhi33347OTk5REdH8/HHHzNq1KjKZbNnz2bChAm8+eabbNmyZb/tt27dSteue2/tpKen8/3331d7rI0bN/Lll1/y008/MXbsWNatW0dKSgpz5swhKiqKtWvXcv7551eWRJk3bx7Lli2rfELphRdeIDExkeLiYo444ggmTZpEUlJStcf69NNPWbt2LfPmzUNVOfPMM/nqq68qk58xZq9AURGF335LwdwvKfjqq8rEsN9VQs+eRPbqSXj37ngiDrWL9UPXZpNFKAwcOJBbbrmFk046ibi4OIYNG0ZYmPNP8MILL3Dttdfy17/+lTPPPJOIaj4c1fU9UtO3inPOOQePx0Pfvn3p1asXq1atomfPnlxzzTUsXrwYr9fLmjVrKtc/8sgj93mU9dFHH+Xdd98FYMuWLaxdu/aAyeLTTz/l8MMPB6CgoIC1a9dasjDGVbZli5McvvySonnz0LIyPLGxxB5zDHHHH0/sMUcTlpLSrB/+aLPJ4kBXAI3p0ksv5dJLLwXgtttuIz3dedhrwIABfPqp8xTDmjVr+Oijj/bbNj09fZ8rjoyMDLp06VLtcap+6ESEhx56iNTUVJYsWUIgECAqKqpyeWxsbOX43Llz+c9//sO3335LTEwMxx9//AEfdVVVbr31Vq644ora3r4xbYKWl1O0aBEFX35JwdwvKfvpJwAievSgwwUXEHf8GGJGjECawRVDXbXZZBEqWVlZpKSksHnzZt555x2+/fbbfeYHAgHuvvturrzyyv22PeKII1i7di0bNmwgLS2N119/nVdffbXa47z55ptcdNFFbNiwgfXr19O/f3/y8vJIT0/H4/Ewc+ZM/H5/tdvm5eXRoUMHYmJiWLVqFd99912161UYN24cd9xxB7/61a+Ii4tj69athIeHk5KScsDtjGlNfLm5FH71lZMg/vs/Avn5EB5O7BGj6HDOZOLGjCGiBXebYMmiiU2aNImcnBzCw8OZMWMGHTo4tRRfe+01ZsyYAcDZZ5/Nb3/7WwC2bdvGZZddxscff0xYWBiPP/4448aNw+/3c8kllzB4cPVXSP3792fMmDFkZmby1FNPERUVxVVXXcWkSZN48803GTt27D5XE8HGjx/PU089xdChQ+nfvz+jR48+4Hs6+eSTWblyJUcddRTgPC77yiuvWLIwrZqqUrp6NQVz51Iw90uKlywBVbzJHYk/+STixowh9uhj8MZV//+spWm1fXCPGjVKq/ZnsXLlSgYOHBiiiJrOxRdfzOmnn84vf/nLUIdSo7byb2Fan0BREbtmziT3jVn4duwAIGrIEOLGjCFuzBiiBg9CPM2lOMbBE5EFqjqq6ny7sjDGmDrQ8nJ2v/022TNm4M/eSexxx5L8hz8Qd9yxhFVTXqi1sWTRCr300kuhDsGYVkNVyf90DtkPPUTZxo1EjxxJyqOPEuM+/ddWWLIwxpgaFP3wA5kPPEDJkh+J6NOb9CeeIG7s8c36EdfGYsnCGGOqKFmzhuwHH6Jg7lzCUlPpPPVu2k2YgIS13VNm233nxhhTRfn27WQ/9jh5772HJzaW5Bv+ROKFFzZKraWWxpKFMabN8+flkfPss+x6+RUIBEi86CKSLv8dYe6j7ab5dH7UZhxqifIePXowZMgQhg8fXllXqilZiXLTmgRKS8l5/nnWnXQyOc+/QML4cfT+5F+k3nKzJYoqLFk0oeAS5UuWLOHDDz9k7dq1AFx22WVMmzaNpUuXctZZZ3H//ffXuJ8vvviCxYsXU/V3JA2hpl91G9OaqN/P7nff46fxp5B1/wNEDxtGz3ffocu99xKeVmPPzW2aJYsmFFyiPCwsrLJEObBfifK333673se5+OKLufLKKzn22GPp168fH374IeBUoj322GMZMWIEI0aM4JtvvgGcWlBjx47lggsuYMiQIQBMnDiRkSNHMnjwYJ555plaj3n//fdzxBFHMHToUO688856x25MY1JV8ufOZcPEs9h+662EJSXR7aUX6fbsM0QNGBDq8Jq1tnvP4l9TYMfSht1npyFwSs291x1qiXJwCgKefPLJiAhXXHFFjf1MWIlyY/ZVum4dO+76K0U//EB4t26kPfQg8ePHt8nHYOuj7SaLEDjUEuUAX3/9NV26dCErK4uTTjqJAQMGVHtSthLlxuxVtHARW668EvF6Sb3jz3SYPLlFVXxtDtpusjjAFUBjOpQS5UBlSfKUlBTOOuss5s2bV+1J2UqUG+Mo+O9/yfjDtYSlptDt+ReISLd7EvVh9yyaWFZWFkBlifLzzz9/n/kHKlFeWFhIfn5+5finn37KYYdV3+Psm2++SSAQ4KefftqnRHnnzp3xeDy8/PLLDVqi/IUXXqDA7SR+69atle/HmFDK+/Ajtvz+KiJ69qTHP/9pieIQtN0rixA5lBLlmZmZnHXWWQD4fD4uuOACxo8fX+1xrES5aet2vfoqmX+7m5iRI0l/8gm88fGhDqlFsxLlrZCVKDdtmaqy84kn2PnY48SNHUvaQw/iCWpyNQdmJcqNMa2eBgJk3jON3Jdfpt2ECXSeenebrufUkOyv2ApZiXLTFml5Odtuu509H3xA4kUXkXLLzS26E6LmxpKFMabFCxQXs/X6P1Lw5ZckX389SVdcbr+faGCWLIwxLZp/zx62/P4qihcupNP//R8dzjs31CG1SpYsjDEtli87m82X/Y7S9etJe+hBEmp4OtAcOksWxpgWqWzLFjZfehm+nTvp+tSTxB1zTKhDatXs7k8TO5QS5atXr2b48OGVr4SEhH320RSsRLlpDkpWr2HjBRcQyMuj+4svWKJoApYsmtChlijv378/ixcvZvHixSxYsICYmJjKH+k1FCtRbpq7ooWL2PTrXyPiofsrLxM9bFioQ2oTLFk0oYYsUf7ZZ5/Ru3dvunfvvt8yK1FuWquCr75i8yWXENahA91ffZXIvn1DHVKb0WbvWdw7715W7VrVoPsckDiAW468pcblDVGivMLrr79eWVeqOlai3LQ2eR9+xLYpU4js15duzz5LWA2fR9M4QnJlISJ/FJHlIrJMRF4TkSgRSRSROSKy1h12CFr/VhFZJyKrRWRcKGJuCMElysePH79fifIZM2YwcuRI8vPzayxRDlBWVsbs2bOZPHlyjetUV6K8vLyc3/3udwwZMoTJkyezYsWKyvWrK1E+bNgwRo8eXVmivCbBJcpHjBjBqlWrDri+MQdr16uvsu2mm4g5/HC6z5xpiSIEmvzKQkTSgGuBQapaLCKzgPOAQcBnqjpNRKYAU4BbRGSQu3ww0AX4j4j0U9VDalw/0BVAYzrUEuUA//rXvxgxYgSpqak1rmMlyk1rsE+dpxNOcOo8RUaGOqw2KVT3LMKAaBEJA2KAbcAEYKa7fCYw0R2fALyuqqWqugFYBxzZtOE2nEMpUV7htddeO2ATFFiJctM67HzySXY+9jjtJk4k/dFHLFGEUJMnC1XdCjwAbAa2A3mq+imQqqrb3XW2AxX1rdOA4Ab8DHdeizRp0iQGDRrEGWecsV+J8n79+jFgwAC6dOmyT4nyU089tXL7oqIi5syZw9lnn33A41SUKD/llFP2KVE+c+ZMRo8ezZo1aw5Yotzn8zF06FDuuOOOOpUov+CCCyof/f3lL39Z2e+GMfVV9MMP7Hx8BglnnkHnv0+1goAh1uQlyt17EW8D5wK7gTeBt4DHVbV90Hq5qtpBRGYA36rqK+7854GPVXW/x4VE5HLgcoBu3bqN3LRp0z7L20pZbCtRblo6f14e6yeehUSE0/Ptd/DGVf/FxjS8mkqUh6IZ6hfABlXNVtVy4B3gaCBTRDoDuMOKdowMoGvQ9uk4zVb7UdVnVHWUqo5KTk5utDdgjGk8qsr2v9yJLzubtAcesETRTITium4zMFpEYoBi4ERgPlAIXARMc4fvu+vPBl4VkQdxbnD3BeY1ddAtiZUoNy1Z3jvvkP/vf5N8w5+Idn/3Y0KvyZOFqn4vIm8BCwEfsAh4BogDZonIpTgJZbK7/nL3iakV7vpXH+qTUMaY5ql0wwZ2TP07MaNHk+Q+NWiah5DcMVLVO4GqP/MtxbnKqG79qcDUxo7LGBM6WlbGthtvwhMeTpd7p1nHRc2MPV5gjGkWsh99lJLly0l//DHCD/AbIhMalrqNMSFX+M035Dz3PO3PO5f4X/wi1OGYaliyaGKHUqL8QNs3FStRbhqaLzeXbbdMIaJ3b1JvCU1lBVM7SxZN6FBLlB9o+4ZiJcpNU1JVtt/+Z/y7d5P2wP14oqNDHZKpgSWLJnSoJcoPtH0wK1FuWordr79Oweefk3LjDUTZjzSbtTZ7g3vH3/9O6cqGLVEeOXAAnW67rcblh1qi/EDbV2Ulyk1zV7p2LZnT7iX22GPp8OtfhzocU4s2myxCIbhEeVxc3H4lyq+99lr++te/cuaZZ1ZbovxA21dVXYnynj17cs0117B48WK8Xi9r1qypXL+6EuUVVy0VJcoPlCwqSpQDFBQUsHbtWksWpkaB0lK23nAjnrg4utzzd3tMtgVos8niQFcAjelQS5TXtH1VVqLcNGdZD0yndM0auj79FGEdO4Y6HFMHls6b2KGWKK9p+6qsRLlprvLnziX35Zfp8JtfEzdmTKjDMXXUZq8sQmXSpEnk5OQQHh6+X4nyGTNmAHD22WfvU6L8sssu4+OPPz7g9lVVlCjPzMzcp0T5pEmTePPNNxk7duwBS5Q/9dRTDB06lP79+9epRPnKlSs56qijAOdx2VdeeYWUlJQDbmfaHl92Nttvu53I/v1JueGGUIdjDkKTlyhvKqNGjdKKm7cV2kpZbCtRbpojDQTY8rvLKZo/n55vv0Vknz6hDslUozmVKDfGtEG7/vEPCr/+mtRbp1iiaIGsGaoVshLlprkpWbGCrOkPEnfiibQ/99xQh2Pqwa4sjDGNKlBczNYbbyKsfXs63/23/Z7UMy1Dm7uyUFX7sIZYa71PZqqXOe1eyjZsoNsLzxNWwwMZpvlrU1cWUVFR5OTk2MkqhFSVnJycfX7jYVqvPXPmsPuNN0i69BJi3aflTMvUpq4s0tPTycjIIDs7O9ShtGlRUVE1/pjQtB7lO3aw4893EDV4MMnXXhvqcMwhalPJIjw8fJ+SFsaYxqF+P9tumUKgvJwuD9yPVFO+xrQsbaoZyhjTNHKef4Gi77+n0+23E2lf0FoFSxbGmAZVvHQp2Y8+Svwp42l39lmhDsc0EEsWxpgGo34/2/9yJ2EdO9L5//7PnjxsRSxZGGMaTN6771K6ciUpN92It127UIdjGpAlC2NMg/AXFJD18CNEH344CaeeGupwTANrU09DGWMaT87Tz+DfuZPUJ5+w5qdWyK4sjDGHrCwjg10vvUS7CROIdvtxN62LJQtjzCHLuv8BCAsj+U9/DHUoppFYsjDGHJKiH34g/9//Jul3lxGemhrqcEwjsWRhjKk39fvZcc89hHXuTJLbu6NpnSxZGGPqLe+99yhdsZKUG27AEx0d6nBMI7JkYYypF39BIVkPPUz0sGEknGaPyrZ29uisMaZecp5xH5Wd8bg9KtsGhOTKQkTai8hbIrJKRFaKyFEikigic0RkrTvsELT+rSKyTkRWi8i4UMRsjNmr4lHZhDPPIHrYsFCHY5pAqJqhHgE+UdUBwDBgJTAF+ExV+wKfudOIyCDgPGAwMB54QkS8IYnaGANA1gPTwesl5U9/CnUopok0ebIQkQTgOOB5AFUtU9XdwARgprvaTGCiOz4BeF1VS1V1A7AOOLIpYzbG7FU0fz75n3xC0mWXEt6pU6jDMU0kFFcWvYBs4EURWSQiz4lILJCqqtsB3GGKu34asCVo+wx3njGmiWkgQObf7yGsUyeSLrkk1OGYJhSKZBEGjACeVNXDgULcJqcaVHfnrNpOtEXkchGZLyLzretUYxpe3nvvU7JihT0q2waFIllkABmq+r07/RZO8sgUkc4A7jAraP2uQdunA9uq27GqPqOqo1R1VHJycqMEb0xbFSgsJOuhB51HZU8/LdThmCZW52QhIhEiMlREhohIvTvUVdUdwBYR6e/OOhFYAcwGLnLnXQS8747PBs4TkUgR6Qn0BebV9/jGmPrZ+eyz+LN3knrrFHtUtg2q0+8sROQ04CngJ5xmoZ4icoWq/quex/0D8E836awHfouTuGaJyKXAZmAygKouF5FZOAnFB1ytqv56HtcYUw/lW7ey64UXSTjjDKKHDw91OCYE6vqjvOnAWFVdByAivYGPgHolC1VdDIyqZtGJNaw/FZhan2MZYw5d1vTp4PGQYlVl26y6NkNlVSQK13r23lMwxrRiRQsXsufjf5F06aWEd+4c6nBMiNT1ymK5iHwMzMJ5Emky8IOInA2gqu80UnzGmBCqfFQ2NZWkS+1R2basrskiCsgExrjT2UAicAZO8rBkYUwrlPf+bEqWLaPLfffiiYkJdTgmhOqULFTVCtUb08YECgvJfvBBooYOJeH000Mdjgmxuj4N9SLV/BBOVe261JhWaudzz+HLzib9sUcRj/Vm0NbVtRnqw6DxKOAsavhhnDGm5at8VPb00+1RWQPUvRnq7eBpEXkN+E+jRGSMCbms6Q+CCCk3WFVZ46jvtWVfoFtDBmKMaR6KFi5iz8cfk3TJJfaorKlU13sW+Tj3LMQd7gBuacS4jDEhoIEAmffcQ1hKCkmXXRrqcEwzUtdmqPjGDsQYE3p7PviAkqVL6XLvNHtU1uzjgMlCREYcaLmqLmzYcIwxoRIoKiJr+oNEDRlCwhlnhDoc08zUdmUx3R1G4dRyWoLTFDUU+B74eeOFZoxpSjnPPY8vK4u0hx+2R2XNfg74iVDVsao6FtgEjHD7ihgJHI7TvakxphUo37aNnOefJ+HUU4kZcXiowzHNUF2/PgxQ1aUVE6q6DBjeKBEZY5pc1oMPAdijsqZGdf1R3koReQ54BedpqAuBlY0WlTGmyRQvWcKeDz8k6corCE+z7u1N9eqaLH4L/B64zp3+CniyUSIyxjQZVSXznml4kzvS8Xe/C3U4phmr66OzJSLyFPCxqq5u5JiMMU1kz8cfU7x4MZ2n3o0nNjbU4ZhmrE73LETkTGAx8Ik7PVxEZjdiXMaYRhYoKSFr+nQiBw6k3cSJoQ7HNHN1vcF9J3AksBsqu0Xt0SgRGWOaxK6XZuLbtp3UKVMQrzfU4Zhmrq7JwqeqeY0aiTGmyfiys8l55hnifnEisT87MtThmBagrje4l4nIBYBXRPoC1wLfNF5YxpjGlPXIIwTKy0m98cZQh2JaiLpeWfwBGAyUAq8CecD1jRSTMaYRlaxaRd7b75B4wQVE9OgR6nBMC1HXp6GKgNtF5O+qWtjIMRljGomqkjntXrzt2tHxqt+HOhzTgtT1aaijRWQF7g/xRGSYiDzRqJEZYxpcwRdfUPTdd3S85hq87dqFOhzTgtS1GeohYByQA6CqS4DjGisoY0zD07Iysu69j4hevehw7jmhDse0MHW9wY2qbhGR4Fn+hg/HGNNYcl97jbJNm+j69FNIeHiowzEtTF2TxRYRORpQEYnAeRrKakMZ00L4cnPJnvEEscccQ+xx1ihgDl5dm6GuBK4G0oCtOBVnr26kmIwxDWznjCcIFBSQcsvNVGkhMKZO6vo01E7gV40cizGmEZSuX0/ua6/RfvJkovr1C3U4poWq69NQvUTkAxHJFpEsEXlfRHo1dnDGmEOXdd/9eKKjSb72D6EOxbRgdW2GehWYBXQGugBvAq81VlDGmIZR+M03FMydS8crryAsKSnU4ZgWrK7JQlT1ZVX1ua+KTpCMMc2U+v1kTruX8PR0OvzmN6EOx7RwdU0WX4jIFBHpISLdReRm4CMRSRSRxPocWES8IrJIRD50pxNFZI6IrHWHHYLWvVVE1onIahEZV5/jGdPW7H7rbUrXrCHlxhvxRESEOhzTwtX10dlz3eEV7L2iEOASd7o+9y+uw3n8NsGdngJ8pqrTRGSKO32LiAwCzsOpTdUF+I+I9FNV+52HMTXwFxSQ/eijRI8cSfy4k0MdjmkF6nplcQswTFV7Ai8CS4BJqtpTVQ86UYhIOnAa8FzQ7AnATHd8JjAxaP7rqlqqqhuAdTh9axhjapDz9NP4c3JInXKLPSprGkRdk8WfVXWPiPwcOAl4iUPrg/th4GYgEDQvVVW3A7jDFHd+GrAlaL0Md95+RORyEZkvIvOzs7MPITxjWq6yjAx2vTSTdhPOJHrIkFCHY1qJuiaLiiaf04CnVPV9oF6NoCJyOpClqgvqukk186q9ua6qz6jqKFUdlZycXJ/wjGnxsqZPB6+X5D/+MdShmFakrvcstorI08AvgHtFJJK6J5qqjgHOFJFTgSggQUReATJFpLOqbheRzkCWu34G0DVo+3RgWz2PbUyrVrRwIfn/+oSOV19NeKdOoQ7HtCJ1PeGfA/wbGK+qu4FE4Kb6HFBVb1XVdFXtgXPj+nNVvRCYDVzkrnYR8L47Phs4T0QiRaQn0BeYV59jG9OaaSBA5j3TCEtJIenSS0IdjmllDqbzo3eCprcD2xs4lmnALBG5FNgMTHaPtVxEZgErAB9wtT0JZcz+9nz4ISVLl9L5nnvwxMSEOhzTyohq6/xt3ahRo3T+/PmhDsOYJhEoLuanU04lLCmJHm/OQjz1bSU2bZ2ILFDVUVXn17k/C2NM85Xzwgv4duwg7YH7LVGYRmGfKmNauPLMTHKee574k08mZtR+XwiNaRCWLIxp4bIfehh8PlJuujHUoZhWzJKFMS1Y8bLl5L33Hh1+82siunatfQNj6smShTEtlKqSNW0a3g4d6HjllaEOx7RyliyMaaHy58yhaP58kq/9A974+FCHY1o5exrKmBbIl5vLjr/9jch+/Wg/eXKowzEh5g/48asfX8CHT33Eh8c3eAFJSxbGtDCqyo6/3Elgdx5dnn0WCbP/xk2tPFBOYVkhJf4SSv2llPicYfB4ib+EMn/ZPtOlvtJ9xivWKfWXVp7s/QE/5YHyvdMVScB9Bc8vD5TjD/jRKuXyFly4gAhvw/ZhYp8yY1qYvHfeJX/OHFJuupGoAQNCHU6L5Qv4yC/LZ0/ZHvaU7iGvLI89pXuc6Zrmle0hrzSPYl9xvY4Z5gkjyhtFpDeSqLAoIrwRRHmdYZgnjAhPBGFhYXg9XsLEHXrCCJMwwjx754d53GnxVs4P94RXTjdGWXpLFsa0IGWbN5M5dSoxRx5J4sUXhzqcZklV2VO2h60FW9lWsI2tBVsrX1lFWZUn/4LyggPuJzosmviIeNpFtiMhIoG0uDQGRgysnI6LiHNO/GGRlSf84OmqCSHSG4nX422iv0LDs2RhTAuhPh/bbr4FvF66TLsH8bbcE8+hKiwvJCM/Y79kUDFdNRHEh8eTFp9Gakwq/Tr0IyEiwXlFOsOKBFA5HdGOcG94iN5d82TJwpgWIufZZylevJguDzxAeJcuoQ6n0ZX4SliRs4LVuav3Swp5pXn7rBsdFk1aXBppcWmMTB1Jl7gupMel0yWuC2nxaSREJNRwFFNXliyMaQGKf/yR7MdnkHDaabQ7/bRQh9MosouyWZy9mEVZi1iStYQVu1bgC/gAiPRGOif+uDSGdBxCWlxa5XRaXBrtI9tb97GNzJKFMc1coKiIbTfdTFhKCp3+ckeow2kQvoCPtblrWZy9mMVZi1mSvYStBVsBJzEMThrMRYMuYnjKcAYlDSI5OtmSQYhZsjCmmcu89z7KNm+m24sv4m3XLtTh1Muesj38mP0ji7MWszh7MUuzl1LkKwIgJTqF4SnD+dXAXzE8eTgDEgfY/YJmyJKFMc1Y/hdfsPuNN0i85BJiR/8s1OHUiaqyOX9zZWJYnLWYn3b/hKJ4xEP/Dv2Z0GcCw5OHMzxlOJ1jO9tVQwtgycKYZsq3cyfbb/8zkf37k3z9daEOp1bbC7bzwfoPmP3TbDbt2QRAfEQ8w5KHMb7HeIanDGdIxyHEhFsvfi2RJQtjmiFVZfuf7yBQUECXl17EE9Gwv8ZtKEXlRfxn83+YvW4283bMQ1FGpY7iN4N+w4iUEfRq3wuPWAm61sCShTHN0O43ZlEwdy6pt91KVL9+oQ5nHwEN8MOOH5j902zmbJpDsa+YrvFd+f3w33NGrzNIj08PdYimEViyMKaZKd2wgcx77yX26KPocOGFoQ6n0qY9m3h/3ft8uP5DthduJy48jlN7nsqZvc/k8JTD7b5DK2fJwphmRMvL2XbzLUhEBJ3vuSfk/WnvKdvDJxs+YfZPs1mSvQSPeDiq81FcP+J6Tuh2AlFhUSGNzzQdSxbGNCM7n3ySkqVLSXv4YcJTU0MSgy/g45tt3zD7p9l8sfkLygJl9G7Xmz+O/COn9zqdlJiUkMRlQsuShTHNRNHCRex86mnaTZxIwvhxTX78NblrmL1uNh9t+IidxTtpH9meSf0mMaH3BAYlDbJmpjbOkoUxzYC/oJBtt9xCeOfOpP759iY99qKsRTy68FHmZ84nTMI4Nv1YJvSewHHpx9mP40wlSxbGNAOZ9/yd8q1b6f6PmXjj4prkmKt2reKxRY/xVcZXJEUlceOoGzmj9xkkRiU2yfFNy2LJwpgQ2/Ppp+S9/Q5JV1xBzKhRjX68DXkbmLF4Bv/e+G8SIhK4fsT1nD/gfPuxnDkgSxbGhFB5VhY7/nInUYMHk3z1VY16rO0F23lyyZO8/9P7RHojuXzo5Vw0+CIr323qxJKFMSGiqmy/7XYCJSV0uf8+pJF+pb2zeCfPLX2OWatnAXDBgAu4bMhlJEUnNcrxTOtkycKYEMn956sU/u9/pP7lDiJ79Wrw/eeV5jFz+UxeWfkKZf4yJvaZyJXDrqRTbKcGP5Zp/SxZGBMCpevWkXX//cQedywdzj+/QfddVF7Eq6te5YVlL5Bfls8pPU/h6uFX0z2he4Mex7QtliyMaWJaVsbWm27GExNDl6lTG+z3C2X+Mt5c8ybP/PgMu0p2cXz68Vxz+DX0T+zfIPs3bVuTJwsR6Qr8A+gEBIBnVPUREUkE3gB6ABuBc1Q1193mVuBSwA9cq6r/buq4jWko2Y89TunKlaTPeJyw5ORD3p8v4OODnz7gySVPsr1wO0d0OoJHDn+E4SnDDz1YY1yhuLLwATeo6kIRiQcWiMgc4GLgM1WdJiJTgCnALSIyCDgPGAx0Af4jIv1U1R+C2I05JIXz5pHz3HO0n/xL4k888ZD2FdAAn276lBmLZrBxz0YOSzqMu46+i9GdR9uvrU2Da/Jkoarbge3ueL6IrATSgAnA8e5qM4G5wC3u/NdVtRTYICLrgCOBb5s2cmMOTdGiRWRcfQ0R3bqROmXKIe1rec5ypn43laU7l9KnfR8eHvswJ3Q9wZJEY/H7wF+2/ysQAPVDwA8Bnzte3Tw/aKDmeeqOq7rDAKBB87TKvEAN83CGx94A3oY9vYf0noWI9AAOB74HUt1EgqpuF5GKamVpwHdBm2W486rb3+XA5QDdunVrpKiNOXiF333PlquuIiy5I91efAFPbGy99pNXmsdjix5j1upZJEYlMvXnUzmt52l4Pd4GjriZCwSgvBBKC6CsEMqChxXjFcuDpssLwV/unOh9pXvHK17VzfOXuSflFuSY61pPshCROOBt4HpV3XOAb0TVLdDqVlTVZ4BnAEaNGlXtOsY0tYIvvyTj2uuI6NaVrs8/T3jKwVdtDWiA99e9z0MLHiKvLI9fDfwVVw2/iviI+EaIuAmpQnEuFO6EwiwoyILC7L3Dwmwo2rV/IigvqvsxvJEQEQsRcRAeDWERzjxvBIRFQlSCM+4Nd4dBr7CI/ed5w53tPOHg8YJ4wBPmjnv3HdZ1nniqvMQZIlXmSTXzPFXWrdi2YYUkWYhIOE6i+KeqvuPOzhSRzu5VRWcgy52fAXQN2jwd2NZ00RpTf3s++Tdbb7qJqL596fr8c4R16HDQ+1i1axV3f3c3S7KXcHjK4dz+s9ub9xNOgQAU7dz3pF+Q5SSDwp17xwvcZBAo338f4oGYjhCXAjGJEJPknvBjITLOOfFXTEfEB43HucuDpq0YYoMIxdNQAjwPrFTVB4MWzQYuAqa5w/eD5r8qIg/i3ODuC8xruoiNqZ/d773H9ttuJ3rYMLo+8zTe+IO7CthTtocZi2bw+urXaR/ZnruPuZszep/RfPq0Li+BXT/BzjWwc607XAM71znNPVV5IyA22XnFpULqEIhLhtgUd547HpcC0YkQ4o6fzL5CcWVxDPBrYKmILHbn3YaTJGaJyKXAZmAygKouF5FZwAqcJ6mutiehTHOX+/rr7Pi/u4g5ajRdZ8zAE1P3In2qyofrP2T6/OnkluZyTr9z+MOIP4SuhlNhDuxcvX9SyN3E3hZhgfZdoWM/6H4MdOi578k/Nhmi2rnNKKYlEtXW2bQ/atQonT9/fqjDMG1QzosvkXXvvcSNGUPao4/giYys87Zrctcw9bupLMxayNDkodz+s9sZlDSoEaN1BQKQu6HKFYI7Xrxr73phUZDUFzr2dRJDx76Q3B8Se0OEVa1tDURkgaruV/7YfsFtTANRVXY+8QQ7H3uc+PHjSbvv3joXBywoK2DG4hm8tuo14iPiuevou5jYZ2LjNTkF/LBjKWz8H2z6GjZ9AyW79y6PTXaSwaAzoWP/vYmhXVdrHmqjLFkY0wBUlezp08l57nnaTZxI57v/hoTV/t9LVfl4w8c8MP8BcopzmNxvMteOuJZ2ke0aNkC/D3YsgY1fOwli83dQmucsS+wFA8+ArkdC8gBI6uPcVDYmiCULYw6RBgJk3j2V3Fdfpf3559HpjjuQOnz7/mn3T0z9fio/7PiBwUmDeeyExzis42ENE5S/HLYthk3/cxLE5u+gLN9ZltQHDjsLuv8cehwDCV0a5pimVbNkYcwhUL+f7X++g7x33yXx0ktIufHGWn9FXVheyFNLnuKVFa8QEx7DX476C2f3OfvQfljnK4Nti2Djf51mpc3f730iqWN/GDoZevzcufkcbyXKzcGzZGFMPWl5OVtvvpn8f31Cxz9cQ8errjpgoghogE82fML0BdPJKspiUt9JXDfiOjpEHfxvL1B1ksO6z5yrh83fg6/YWZYyCIZf4Fw1dD/GeRrJmENkycKYegiUlrL1+j9S8MUXpNx8M0mX/PaA6/+w4wemz5/O8pzlDEwcyIPHP8iw5GEHd9CKBLH8XVjxPuze5MxPPQxGXuQkhu7HQKz1gGcaniULYw5SoKiIjGuuofCbb+n0f3fS4bzzalx3fd56HlrwEHO3zCU1JpWpP5/K6b1Or/tTTqqwbSEsfw9WvAe7NzulJXodD8fdBP1PteRgmoQlC2MOgj8/ny1XXEnx4sV0nnYP7SdOrHa9nOIcnlzyJG+teYuosCiuG3EdFw68kKiwqNoPUpkgKq4gKhLEWBhzi5Mg7Gkl08QsWRhTR77cXLZc9jtKVq8m7cEHSRg/br91in3FvLLiFZ5f9jwlvhIm95vMlcOuJCm6lm//qrB1IaywBGGaJ0sWxtSBLzubzZdcStmmTaQ//hjxxx+/z3J/wM+H6z/k0UWPklWUxQldT+D6kdfTs13PmndakSCWvwMrZkPeZqeSae+xMGYKDDgVoutx89uYRmDJwphalG/bxubfXkJ5djZdn3ma2NGj91n+7bZvmT5/OqtzV3NY0mHcd9x9jEwdWf3OVGHrAreJqUqCON4ShGm+LFkYU4NASQm7Zv6DnKefBq+Xbs89R8yIwyuXr81dy/QF0/l669ekxaVx33H3Ma7HuOpvXu9cBz++4bx2b3ITxAkw9lbof4olCNPsWbIwpgpVZc9HH5P14HR827YTd+KJpN50IxE9egCQVZTFjMUzeG/de8SGx3LjqBs5f8D5RHir1IEq3AnL3nYSxNYFTh8NPcc49yAGnAbR7Zv8vRlTX5YsjAlStGgRWdPupXjJEiIHDqTL3+8hdvTPnGXlRby0/CVeWv4S5YFyfjXwV1w+5HLaR7Xfu4PyYlj9MSx5A9b9x+lbOXUInHw3HPZLSOgcmjfWAgUCSpk/QLk/QCAAflX8AfelSqDquCo+vxJw13OG7LuNKqqKKgQUd9r5ghBQUNxh5Tr7Dvcud+eBM4Hb/XXFtu57cLZx5rHPPK1cFrQLlH3Xq6piP8HLqu4D4OqxvQnzNmzBR0sWxgBlGVvJfnA6ez7+F2HJyXT++99pN+FMxOvFF/Dx3rr3mLF4BjuLdzKuxziuO/w6uia4HTgGAk6ZjR9nOU8yleVDfBc4+hoYei6kDg7tmzsEqkqpL0BhqY+iMj/F5X4KS30Ul/kpKvNTWLZ3vKjM5w79lJT73RO9Uu5zTvgVJ/5yvzrTvqB5Pt1vHX+gdXaf0BSuGNOLsAbult2ShWnT/AUF5Dz9DLtmzgSPh45XXUXSpZfgiY0lpziHd9e9y1tr3mJrwVaGJw/noeMfYnjKcGfjzBXw4+uw9C3Ys9Xp3nPQBBh6jlOH6VBqPTUgf0DZXVRGblEZuwrL2VVYMV5GbmEZuUXl5BaVkVdc7iSCcvfkX+qjqNxf7TfcmoR7hehwL1HhXiLCPER4PYR7PUSEeQj3CuFeDzERXsK94ZXTFeuEh1WZrpjn8eDxCF4Br0fwejx4PeARcacFjwhhHnHXc+e54x4PhLnbiDjresTZHpyhxwOCM9/p6tpZT9zlFd1fV4xXLEOc7SqqvAjOts6wYp446zkDZ17QOu6equ0Xap997DcveD3ZZ15j9DFlycK0Serzsfutt8l+9FH8u3bRbsKZJP/xj4SlprIwayFvLHiDOZvm4Av4OLLTkdx0xE2c0PUEpCATvnnMaWbKXArihT6/gJP/Bv1OaZIOgFSV3UXlbN1dzI68EnYVOSf9ymGhc/KvmJdXXF7jCT8mwkuHmAgSYyNIiA4jMTaGmAgvMRFhxER4iY3wEu2OB8+vGI+O8BIb6SUm3BmPCLO+LlorSxamzSn47//Iuu9eSteuI3rUSFKffhp//x68tf4DZn0/i3W71xEfHs+5/c/lnH7n0Cs6GVZ9BHPPgg1fggagywg45T4YfLbTfWgDUlVyCsvIyC1ma24xGblFzvhuZ3xrbjGFZfv3LBzh9ZAYG0GH2AgSY8MZ1CXBmXaTQYfYCBJjIugQG145Pyq8eVz9mObPkoVpM0rXrSPzvvso/Oq/hHftStqjj7B1ZFfuWzOLj978iGJfMYOSBnHX0XcxPnkkMT99AR/dAuvngr8M2neDY29w7kN07FvvOAIBZWdBKVvcROAkgb2JYevuYkrKA/tskxAVRnqHGLonxXJMn46kd4ghrX00ndtFkRjrJIOYCG+t5dGNqS9LFqbV8+3aRfZjj7F71pt4YmJIvOlPzDsmiTvXz+THD38k0hvJKT1P4dxOx3DYjjXw1dOw5XtAoX13OOJ3Tk9y3UYfVGNwmS/AxpxC1mYWsCYzn3VZznBTThFl/n2TQWJsBGnto+mXGs/Y/imkd4h2EkKHaNI6RJMQFd7AfxVjDo4lC9NqBcrKyH35ZXY++RSB4mLCzj6Nj8fG82bWTPK+z6NHQndu7ncBZxYU0W7xHMh+1Nmw01A4/lbntxCpg2tNEGW+ABt2FrImM5+1WQWsdYcbdxbic5/oEYHuiTH0SYnnhAFVkkH7aGIj7b+iad7sE2panfKsLAo+/5yc556nPCODkiMH8/pJUXwc+BhvhpcTEodwbmQYR274HlnyX+cmdfejYeS9TrmN9t2q3W+pz+8mhQLWZeazJrOAtVn5bMwpqnzM0yPQPSmWPilxjBucSt+UePqmxtE7Oc7uD5gWzZKFafFUlbKffiL/s8/J//wzSpb8CEBBt468dFEiX3VZTYo3gavC0pm0ZQUp62dDWDT0ORFOuAP6jdunoquqkpFbzMrte1i5PZ9VO/aw2m0+qpoU+qbEccphnembGkfflHh6JcdaUjCtkiUL0yKp30/xokWVCaJ802YAcnp04PtftOOz7gVsSc7laE8cD2ftZkzBZsKiE6H/aU7zUq+xEBFDQamP1TvyWbl9E6t27GHV9nxW7cinoNRXeaweSTH0S43nVEsKpg2zZGFajEBREQVff03BZ5+TP/cLArvz8Id5WNcriq/GeZjfV/DFF3FkqY8L83M5NqOYrnFpcNiFBPqdyqb4oazMLGLllnxW/bCCldvz2byrqHL/8ZFhDOgcz9kj0hjQKYEBnePpnxpv9xOMwZKFaeZ8O3eS/8UX5P1nDkXffoeUlVMc5WF+b+WH4z2s7ikM0nx+VlzEpXtK6F/eCX/nUWxPG8KSsMN4srAzK9fns/qbfIrK/gc4TUg9OsYyJK0dk0emM7CzkxjS2kfbo6fG1MCShWl2StevZ/ecT8n+9CM8K9YhCtnt4IehwsI+HiKTyziivIQrypRu5f3YFj2YJeH9eDCsB/N3RZKXWe7uyUe76B0M7BzPOaO6MrBzPAM6JdAvNZ7oCGtCMuZgWLIwIaOq+DIzKV23jl2rfmT36mX4flhI9I48ADZ2ggXHCLk9fXSPLWFIII4jA31Zkt+XTwu6Md3XDd8u5yOcEh9J7+Q4zhgWS+9k5+mjPilxdG4XZVcLxjQASxam0anfT/mWLeSuXkr2ikUUrlqKbtxC9I58Ikr3/jitNArWdRY2nRggIt1HgjeRdgV9yCgcwD/z+7Lb057uSTH07hLH6JQ4LkyOo3dKHL2SY+1Ha8Y0MksWpsEEysrYs3YlO378ht3L51O2fgPerbtI2FlKmFvKyAv44mBrEuQNUsraB/DFeymJi6XU25H80p5s8gwlkDCIHint6Z0cx+TkWKakxNEtMYbwBq7Rb4ypG0sWplaqSvnuXPK2byRvyyryt69nz/YMSnZm48/djWfnbuKyS+iwO4DHrW4aD2S1h8xEWJOuFCd4KYqPZXdsMr7InoRF9UPa9yEyqSud2sfSp100ndpF0bVDNMnxkdZ0ZEwz02KShYiMBx7B+XL6nKpOC3FILVbA5yM/cyNZG5axc/Na9mzfTGlOFv7cXMgvwFtQQkRROZFFfmKKlZhiCAsqZRTlvgCKIiA3HrZ3hLV9vBS2i6U4sROB1AHEJw0jPmUgKR070aldFF3aR5EcF9ngPXgZYxpfi0gWIuIFZgAnARnADyIyW1VXhDayQxMIBCgvLqSoeA8lBXsoKy6gpDifsqJCSosL8ZUVUl5SjK/UeZUXFVBelI+/qAB/cRFaUoyWliClZUhZOVLuw1vmx1MewFseIMynhJcrYeVKuA8iyiG8HCKCqlu3d18VCiOhIAaKo2F3gpCZ6qU8Opzy6Ej8sbFoXHs87ZOJSOxCdGpvOnTsTWJyDwYntbNEYEwr1iKSBXAksE5V1wOIyOvABKDBk8UbvxlN+J5iRBUCIKqIBg2djnoRBY87DwVPAHdc8bjLJQAeBa8fvAFnHa/f+ZZeMaxOGAf3DxMASiOgPAzKwqE8HHxhUB4ulMYK/nAv/nAPgXAP/vAwNDycQHQUEt8Ob/uORHXsQlzn3iR2G0hqam/6J8QTFxlmTUHGmEotJVmkAVuCpjOAn1VdSUQuBy4H6Nat+mJwtYnNzCchL4AKla+Ax+103SP7zHeWOfN8lfM8zrBiXY84L6+gHg/qDX55oeIV5gVPGISH4QkLh7AwJCwCCQ/HExaBNzyCsJg4IuISiIrvSEy7JGI7pJLQsQvtkzoRE2U/KDPGNJ6WkiyqOwvu11Gkqj4DPAMwatSoevX2fvq/l9dnM2OMadVaSgNzBtA1aDod2BaiWIwxps1pKcniB6CviPQUkQjgPGB2iGMyxpg2o0U0Q6mqT0SuAf6N8+jsC6pq7UXGGNNEWkSyAFDVj4GPQx2HMca0RS2lGcoYY0wIWbIwxhhTK0sWxhhjamXJwhhjTK1EtV6/XWv2RCQb2BTqOICOwM5QB1ENi+vgWFwHx+I6OM0pru6qmlx1ZqtNFs2FiMxX1VGhjqMqi+vgWFwHx+I6OM01rmDWDGWMMaZWliyMMcbUypJF43sm1AHUwOI6OBbXwbG4Dk5zjauS3bMwxhhTK7uyMMYYUytLFsYYY2plycIYY0ytLFk0ARHpJSLPi8hboY4lmIgMFJGnROQtEfl9qOOpICLHi8h/3diOD3U8FUTkWDem50Tkm1DHU0FEBonILBF5UkR+2Qzi2efz3lw+/9XE1Sw+/9XE1Sw//5YsGpCIdBWRL0RkpYgsF5HrAFR1vape2gzjWqmqVwLnAE3+g6Ca4sLpMrcAiMLpJbFZxKWq/3X/Xh8CM5tLXMApwGOq+nvgN6GOp+rnvak//wcRV5N+/g/i/BDSz3+NVNVeDfQCOgMj3PF4YA0wKGj5W80tLuBM4BvgguYSF+Bx56UC/2wucQUtnwUkNJe4gBRgBnA/8HWo4wla/laV9Zvk838wcTXl57+ucYX681/Ty64sGpCqblfVhe54PrASSAttVAeOS1Vnq+rRwK+aS1yqGnBXyQUim0tcACLSDchT1T3NJS5VzVLVq4EpNGF9oZb4ea9m3Sb7/Nc1rlB//mvSYnrKa2lEpAdwOPC9iCQBU4HDReRWVb2nmcR1PHA2zgcypL0QVonrbGAc0B54PIRh7ROXO+tS4MWQBeSq8vfqAdwGxOJcXYQ6nn0+7zg/OAvJ57+WuL4lRJ//WuJaTTP5/O8j1Jc2rfEFxAELgLNDHYvFZXG1tXgsrsZ5WTNUAxORcOBtnLbGd0IdTwWL6+BYXC0zngoWV8Ozch8NSEQE5ymZXap6fYjDqWRxHRyLq26aWzwVLK7GYcmiAYnIz4H/AkuBiptUt6lqqO8HWFwHweJqmfFUsLgahyULY4wxtbJ7FsYYY2plycIYY0ytLFkYY4yplSULY4wxtbJkYYwxplaWLIwxxtTKkoVpk0Rkrog0RVnqa92S1P9s7GMZ05iskKAxB0lEwlTVV8fVrwJOUdUN9TiO4PwWKlDryk2oucZlGpddWZhmS0R6uN/Kn3U7i/lURKLdZZVXBiLSUUQ2uuMXi8h7IvKBiGwQkWtE5E8iskhEvhORxKBDXCgi34jIMhE50t0+VkReEJEf3G0mBO33TRH5APi0mlj/5O5nmYhc7857CugFzBaRP1ZZ/2IReV9EPhGR1SJyZ5X3/ASwEOgqIve7+10qIucG7eNmd94SEZnmzuvt7nOBOL2tDXDnT3b3sUREvnLnDRaReSKyWER+FJG+B3gv1cX1UlBc+7w/0wqFupKhvexV0wvoAfiA4e70LOBCd3wuMMod7whsdMcvBtbhdC6TDOQBV7rLHgKuD9r+WXf8OGCZO/73oGO0x+mgJtbdbwaQWE2cI3FKOMTiVBRdDhzuLtsIdKxmm4uB7UASEA0sw+mtrQdOKYjR7nqTgDmAF6cznM04neicgtNpT4y7XqI7/Azo647/DPjcHV+K0/cFQHt3+BjwK3c8wo2j2vdSTVwjgTlB76d9qD8v9mrcl11ZmOZug6oudscX4Jy0avOFquarajZOsvjAnb+0yvavAajqV0CCiLQHTgamiMhinIQSBXRz15+jqruqOd7PgXdVtVBVC4B3gGPrEOccVc1R1WJ3m5+78zep6ndB+35NVf2qmgl8CRwB/AJ4UVWL3PewS0TigKOBN934n8ZJLABfAy+JyO9wEg84/TncJiK3AN3dOA70XoLjWg/0EpHHRGQ80OSdQZmmZfcsTHNXGjTux/n2C84VR8WXnagDbBMImg6w72e+amE0BQSYpKqrgxeIyM+AwhpilJqCr0V1x6fKcWrat1SzvQfYrarD9zuQ6pXuezgNWCwiw1X1VRH53p33bxG57ADH2ycuVc0VkWE4nfRcjdOP9SUH2Na0cHZlYVqqjThNIQC/rOc+zoXKaqB5qpoH/Bv4g3sTFxE5vA77+QqYKCIxIhILnIVTXbQ2J4lIonsfZiLOt//q9n2uiHhFJBmnyWwezn2TS0Qkxo0zUZ2uXjeIyGR3nrgndESkt6p+r6p/wel6tauI9ALWq+qjwGxgaF3fi4h0xOkr+m3gDmBEHd6vacHsysK0VA8As0Tk18Dn9dxHroh8AySw91vx34CHgR/dhLEROP1AO1HVhSLyEs5JHOA5VV1Uh+P/D3gZ6AO8qqrzxeluM9i7wFHAEpwriZtVdQfwiYgMB+aLSBlOt6C34fQl/aSI/BkIB153t73fvYEtOPc1luD02X2hiJQDO4C/us1Z+72XauJKA14UkYovnLfW4f2aFsxKlBsTAiJyMc4N+mtCHYsxdWHNUMYYY2plVxbGGGNqZVcWxhhjamXJwhhjTK0sWRhjjKmVJQtjjDG1smRhjDGmVpYsjDHG1Or/Aasf9LKMiO0XAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "A = {}\n",
    "for p in [ 0.99, 0.995, 0.997, 0.999 ]:\n",
    "    i=\"%d\" %p\n",
    "    A[i] =np.vectorize( lambda n: 1/t(p,n) )(nproc)\n",
    "    plt.plot(nproc,A[i], label=f\"{p*100} parallel\" )\n",
    "\n",
    "plt.xscale('log',base=2)\n",
    "plt.xlabel(\"number of processors\")\n",
    "plt.ylabel(\"speedup\")\n",
    "plt.title(\"Amdahl's law with p>99%\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7653b5af",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Serial Example\n",
    "---------------------\n",
    "\n",
    "Below is a simple example of a python snippet that we want to accelerate.  It does a math function and calculates \"tot\" over a range of numbers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "70f5c093",
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " total is: 1414.21661385363\n",
      "CPU times: user 161 ms, sys: 0 ns, total: 161 ms\n",
      "Wall time: 160 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "max=1000000\n",
    "counter=0\n",
    "tot=1\n",
    "\n",
    "def count():\n",
    "   global tot,counter\n",
    "   counter=0\n",
    "   while counter < max:\n",
    "      tot += 1/tot\n",
    "      counter += 1\n",
    "\n",
    "count()\n",
    "print( f\" total is: {tot}\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "511d4722",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "So it calculates the answer, and it does it in 122 milliseconds, entirely in user space (0 ns of system time.)\n",
    "We want to make it faster, so let's add threading.\n",
    "\n",
    "* import threads\n",
    "* run four threads in parallel.\n",
    "\n",
    "should finish four times faster, right?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a8c41990",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " total is: 2449.49161668406\n",
      "CPU times: user 404 ms, sys: 3.8 ms, total: 408 ms\n",
      "Wall time: 405 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import threading\n",
    "\n",
    "max=1000000\n",
    "tot=1\n",
    "\n",
    "def count():\n",
    "   global tot\n",
    "   counter=0\n",
    "   while counter < max:\n",
    "      tot += 1/tot\n",
    "      counter += 1\n",
    "\n",
    "threadMax=4\n",
    "threads = []\n",
    "threadNumber=1\n",
    "while threadNumber < threadMax:\n",
    "   t = threading.Thread(target=count)\n",
    "   threads.append(t)\n",
    "   threadNumber += 1\n",
    "\n",
    "for t in threads:\n",
    "    t.start()\n",
    "\n",
    "for t in threads:\n",
    "    t.join()\n",
    "\n",
    "print( f\" total is: {tot}\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84d5090d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\n",
    "OK, so the answer is wrong, and it took 3 times more cpu time to do it. How come?\n",
    "\n",
    "* Because the overhead for setting up the threads is much more than the size of the problem. (amortization)\n",
    "* But the answer is still wrong. Why?\n",
    "  * the four tasks are changing the same counters and writing on eachothers work, \n",
    "  * it is a pile of race conditions. \n",
    "  \n",
    "* How can we get the parallel code to get the right answer?\n",
    "\n",
    "* We add locks, to serialize access to the variables the tasks are contending for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9910d980",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " total is: 1414.2180280637874\n",
      "CPU times: user 2.7 s, sys: 2.3 s, total: 5 s\n",
      "Wall time: 2.78 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import threading\n",
    "\n",
    "max=1000000\n",
    "counter=0\n",
    "tot=1\n",
    "counterLock = threading.Lock()\n",
    "totLock = threading.Lock()\n",
    "\n",
    "def count():\n",
    "   global counter,tot\n",
    "   while counter < max:\n",
    "      totLock.acquire()\n",
    "      tot += 1/tot\n",
    "      totLock.release()\n",
    "\n",
    "      counterLock.acquire()\n",
    "      counter += 1\n",
    "      counterLock.release()\n",
    "\n",
    "threadMax=4\n",
    "threads = []\n",
    "threadNumber=1\n",
    "while threadNumber < threadMax:\n",
    "   t = threading.Thread(target=count)\n",
    "   threads.append(t)\n",
    "   threadNumber += 1\n",
    "\n",
    "for t in threads:\n",
    "    t.start()\n",
    "\n",
    "for t in threads:\n",
    "    t.join()\n",
    "\n",
    "print( f\" total is: {tot}\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f721701a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "OK, now instead of 3 times slower it is now almost 20 times slower, and spending half the time in the kernel,\n",
    "but the answer is correct. Why did synchronizing access to get the correct the answer make the cpu time double? \n",
    "\n",
    "**locking is expensive** To parallelize code, use locks sparingly.  locks are signs of serial code, and serial code fundamentally limits performance. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f978a69e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " total is: 1414.2180280637874\n",
      "CPU times: user 687 ms, sys: 810 ms, total: 1.5 s\n",
      "Wall time: 917 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "import threading\n",
    "\n",
    "max=1000000\n",
    "counter=0\n",
    "tot=1\n",
    "critLock = threading.Lock()\n",
    "\n",
    "def count():\n",
    "   global counter,tot\n",
    "   while counter < max:\n",
    "      critLock.acquire()\n",
    "      tot += 1/tot\n",
    "      counter += 1\n",
    "      critLock.release()\n",
    "\n",
    "threadMax=4\n",
    "threads = []\n",
    "threadNumber=1\n",
    "while threadNumber < threadMax:\n",
    "   t = threading.Thread(target=count)\n",
    "   threads.append(t)\n",
    "   threadNumber += 1\n",
    "\n",
    "for t in threads:\n",
    "    t.start()\n",
    "\n",
    "for t in threads:\n",
    "    t.join()\n",
    "\n",
    "print( f\" total is: {tot}\" )\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28d52604",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Here, instead of using a lock for each variable, we use the concept of a shared lock for when both variables.  \n",
    "It eliminates about 40% of the execution time, so now it is only 9x slower than serial."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "628b9fd7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## NEWS FLASH: Serial 9x Faster than Parallel!\n",
    "\n",
    "* Sometimes the thing you are trying to do is inherently serial. \n",
    "* Do something else instead.\n",
    "* more hardware can slow things down because of co-ordination.\n",
    "* locks are hard to get right, and can super easily kill performance.\n",
    "* Also: headlines never tell the whole truth.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a8ff838",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Summary\n",
    "--------------\n",
    "\n",
    "* Application design limits speedup.\n",
    "\n",
    "* **Perfectly Parallel** is the gold standard.\n",
    "\n",
    "* *almost all parallel*, counts for little when things scale.\n",
    "\n",
    "* Locks, joins, waits... they mean you failed to do the above.\n",
    "\n",
    "* (Synchronous) API calls... mean you failed to do the above.\n",
    "\n",
    "* Every small percentage of synchronization or global state, limits ultimate performance. \n",
    "\n",
    "* As the number of processes and processors rises, these small things dominate.\n",
    "  \n",
    "* locks, joins, waits... are all expensive, in terms of cpu time. They add overhead to any application. If you use them, make sure they are worth it, and amortize them by making them as coarse as possible.\n",
    "\n",
    "* Ideally, try to formulate the algorithm so there are no synchronization points.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0f6f04d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Relevance?\n",
    "\n",
    "Note that on our main cluster (ddsr.cmc) the configuration is 128 processors:\n",
    "\n",
    "* 16 processors / node\n",
    "* 8 nodes in the cluster\n",
    "* 600 configurations/node (identical on all nodes)\n",
    "* ~900 processes/node = 7200 processes in the application.\n",
    "* at 7200 processes, 99.9% parallel isn't good enough to make good use of hardware.\n",
    "* At this scale, you really want p to be around 99.99%...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "226c39b7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## So Sarracenia is Not a Parallel App!\n",
    "\n",
    "\n",
    "It is just an ordinary python script that launches processes. It's not a real parallel app.\n",
    "\n",
    "* it is not based on threads !\n",
    "* it does not use global locks, no joins, no synchronization!\n",
    "* it is not leveraging multiprocessing!\n",
    "* it is launching completely independent processes that never synchronize!\n",
    "\n",
    "### Yes, Exactly."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9b00250",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "## THANKS!\n",
    "\n",
    "\n",
    "light reading:\n",
    "\n",
    "* https://en.wikipedia.org/wiki/Amdahl%27s_law\n",
    "* https://jenkov.com/tutorials/java-concurrency/amdahls-law.html\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}