Skip to content

A
Assignment
Commit 1d2f2109 by felix
Options

not fully finished but I want to submit

parent 19639e96
Showing with 150 additions and 8 deletions
homework1/MiniHomework#1.ipynb
......@@ -18,6 +18,7 @@
"\n",
"\n",
"\n",
"\n",
"\n"
]
},
......@@ -25,6 +26,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"$ \\mathbf{w} \\cdot \\mathbf{x} + b = y $\n",
"can be transformed into: \n",
"initially, both $\\mathbf{w}$ and $\\mathbf{x}$ are n-d vector, no we add "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### (a) ```编写逻辑回归的目标函数```\n",
"请写出目标函数（objective function）, 也就是我们需要\"最小化\"的目标（也称之为损失函数或者loss function)，不需要考虑正则。 把目标函数表示成最小化的形态，另外把$\\prod_{}^{}$转换成$\\log \\sum_{}^{}$\n"
]
......@@ -33,7 +43,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"$L(w)=$"
"\n",
"$ L(w)=\\sum_{i=1}^{n}-y_{i} \\log \\left(p\\left(\\mathbf{x_{i}} ; \\mathbf{w}\\right)\\right)-\\left(1-y_{i}\\right) \\log \\left(1-p\\left(\\mathbf{x_{i}} ; \\mathbf{w}\\right)\\right)$\n",
"\n",
"\n",
"where $ p(c) = \\frac{1}{1+e^{-\\mathbf{w}^{T}\\mathbf{x} }}$"
]
},
{
......@@ -48,7 +62,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"$\\frac{\\partial L(w)}{\\partial w}=$"
"$$\n",
"\\frac{d f}{d \\mathbf{X}}=\\left(\\frac{\\partial f}{\\partial x_{1}}, \\frac{\\partial f}{\\partial x_{2}}, \\cdots, \\frac{\\partial f}{\\partial x_{n}}\\right)^{T}\n",
"$$\n",
"\n",
"$\\frac{\\partial L(\\mathbf{w})}{\\partial w_j}= \\frac{1}{m}\\sum_{i=i}^{m}[p(\\mathbf{x}^i;\\mathbf{w}) - y^i]x_j^i $\n",
"\n",
"\n",
"\n",
"$\\frac{\\partial L(\\mathbf{w})}{\\partial \\mathbf{w}}=\\left(\\frac{\\partial L(\\mathbf{w})}{\\partial w_1}, \\frac{\\partial L(\\mathbf{w})}{\\partial w_2}, \\cdots, \\frac{\\partial L(\\mathbf{w})}{\\partial w_n}\\right)^{T}$"
]
},
{
......@@ -63,7 +85,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"$\\frac{\\partial^2 L(w)}{\\partial^2 w}=$"
"$\\frac{\\partial^2 L(w)}{\\partial^2 w}= H$\n",
"\n",
"$H_{ab} = \\frac{\\partial^2 L(w)}{\\partial w_a \\partial w_b}= \\frac{1}{m} \\sum_{i=1}^{m} p(\\mathbf{x}^i;\\mathbf{w})(1-p(\\mathbf{x}^i;\\mathbf{w})) x_a^i x_b^i $ \n"
]
},
{
......@@ -102,10 +126,19 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"// TODO: your proof\n",
"PROOF since \\(p \\geq 1,\\) the function \\(x \\mapsto|x|^{p}\\) is convex. It follows that\n",
"$$\n",
"\\(\\begin{aligned}\\|(1-\\lambda) f+\\lambda g\\|_{p}^{p} &=\\int_{X}|(1-\\lambda) f+\\lambda g|^{p} d \\mu \\\\ & \\leq \\int_{X}\\left((1-\\lambda)|f|^{p}+\\lambda|g|^{p}\\right) d \\mu=(1-\\lambda)\\|f\\|_{p}^{p}+\\lambda\\|g\\|_{p}^{p} \\end{aligned}\\)\n",
"$$\n",
"\n",
"\n",
"\n"
"for any measurable functions $f$ and $g$ and any $\\lambda \\in[0,1]$. In particular, this proves\n",
"that $||(1-\\lambda) f+\\lambda g||_{p} \\leq 1$ whenever \n",
"$ ||f||_p = ||g||_p=1 $\n",
"From this Minkowski's inequality follows. First, observe that if $||f||_p=0$, then\n",
"f=0 almost everywhere, so Minkowski's inequality follows in this case. A similar\n",
"argument holds if $||g_{p}||=0$, so suppose that $||g||_p>0$ and $||g||_p>0$ . Let $\\hat{f}=f /\\|f\\|_{p}$\n",
"and $\\hat{g}=g /\\|g\\|_{p}$, and note that $||\\hat{f}||_{p}=||\\hat{g}||_{p}=1$ . Then\n",
"$$\\(\\frac{\\|f+g\\|_{p}}{\\|f\\|_{p}+\\|g\\|_{p}}=\\left\\|\\frac{\\|f\\|_{p}}{\\|f\\|_{p}+\\|g\\|_{p}} \\hat{f}+\\frac{\\|g\\|_{p}}{\\|f\\|_{p}+\\|g\\|_{p}} \\hat{g}\\right\\|_{p} \\leq 1\\)$$"
]
},
{
......@@ -130,7 +163,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"```(a)``` 请写出linear programming formulation。 利用标准的写法(Stanford form)，建议使用矩阵、向量的表示法。 "
"```(a)``` 请写出linear programming formulation。 利用标准的写法(Stanford form)，建议使用矩阵、向量的表示法。 \n"
]
},
{
......@@ -139,7 +172,17 @@
"source": [
"// your formulation\n",
"\n",
"$\\mathbf{b}_{1}$: Beijing -> 1, $\\mathbf{b}_{3}$: Beijing -> 3, $\\mathbf{s}_{1}$: Shanghai -> 1, $\\mathbf{s}_{3}$: Shanghai -> 3, \n",
"\n",
"Min $$\\mathbf{w}_b^T \\mathbf{b} +\\mathbf{w}_s^T \\mathbf{s} $$, where $\\mathbf{w}_b = [5,6,4]$,$\\mathbf{w}_s = [6,3,7]$\n",
"\n",
"s.t. $$ \\mathbf{b}_1 +\\mathbf{b}_2+\\mathbf{b}_3 \\leq 300 $$\n",
"$$ \\mathbf{s}_1 +\\mathbf{s}_2+\\mathbf{s}_3 \\leq 500 $$\n",
"$$\\mathbf{b}_1 +\\mathbf{s}_1 \\geq 200 $$\n",
"$$\\mathbf{b}_2 +\\mathbf{s}_2 \\geq 300 $$\n",
"$$\\mathbf{b}_3 +\\mathbf{s}_3 \\geq 250 $$\n",
"\n",
" \n",
"\n"
]
},
......@@ -177,6 +220,105 @@
"source": [
"// your formulation"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"from cvxopt import matrix, solvers\n",
"A = matrix([ [1.0, 0.0, -1.0, 0.0, 0.0], [1.0, 0.0,0.0, -1.0, 0.0],[1.0, 0.0,0.0, 0.0, -1.0],[0.0, 1.0,-1.0, 0.0, 0.0],[0.0, 1.0,0.0, -1.0, 0.0],[0.0, 1.0,0.0, 0.0, -1.0] ])\n",
"b = matrix([300.0,500.0,-200.0,-300.0,-250.0])\n",
"c = matrix([1.0,1.0,1.0,1.0,1.0,1.0])\n",
"# sol=solvers.lp(c,A,b)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(5, 6)\n",
"(5, 1)\n",
"(6, 1)\n"
]
}
],
"source": [
"print(A.size)\n",
"print(b.size)\n",
"print(c.size)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "Rank(A) < p or Rank([G; A]) < n",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-36-2fb255c597fe>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msol\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msolvers\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mc\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m/usr/local/lib/python3.7/site-packages/cvxopt/coneprog.py\u001b[0m in \u001b[0;36mlp\u001b[0;34m(c, G, h, A, b, kktsolver, solver, primalstart, dualstart, **kwargs)\u001b[0m\n\u001b[1;32m 3008\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3009\u001b[0m return conelp(c, G, h, {'l': m, 'q': [], 's': []}, A, b, primalstart,\n\u001b[0;32m-> 3010\u001b[0;31m dualstart, kktsolver = kktsolver, options = options)\n\u001b[0m\u001b[1;32m 3011\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3012\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.7/site-packages/cvxopt/coneprog.py\u001b[0m in \u001b[0;36mconelp\u001b[0;34m(c, G, h, dims, A, b, primalstart, dualstart, kktsolver, xnewcopy, xdot, xaxpy, xscal, ynewcopy, ydot, yaxpy, yscal, **kwargs)\u001b[0m\n\u001b[1;32m 571\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mkktsolver\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdefaultsolvers\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 572\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mKKTREG\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mb\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mcdim_pckd\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 573\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Rank(A) < p or Rank([G; A]) < n\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 574\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mkktsolver\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'ldl'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 575\u001b[0m \u001b[0mfactor\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmisc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkkt_ldl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdims\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mA\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkktreg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mKKTREG\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mValueError\u001b[0m: Rank(A) < p or Rank([G; A]) < n"
]
}
],
"source": [
"sol=solvers.lp(c,A,b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
......@@ -195,7 +337,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.1"
"version": "3.7.5"
}
},
"nbformat": 4,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment