20200331

homework1/MiniHomework#1.ipynb

@@ -225,7 +225,7 @@
    "source": [
     "// your proof of second order convexity\n",
     "\n",
-    "1. sufficiency, to prove $\\nabla^{2} f(\\mathbf{x}) \\succeq 0 \\Rightarrow f(\\mathbf{y}) \\geq f(\\mathbf{x}) + {\\nabla f(\\mathbf{x})}^{\\text{T}}$\n",
+    "1. sufficiency, to prove $\\nabla^{2} f(\\mathbf{x}) \\succeq 0 \\Rightarrow f(\\mathbf{y}) \\geq f(\\mathbf{x}) + {\\nabla f(\\mathbf{x})}^{\\text{T}} (\\mathbf{y} - \\mathbf{x})$\n",
     "\n",
     "using tayler expansion at $\\mathbf{x}$, let $\\mathbf{y} = \\mathbf{x} + \\Delta \\mathbf{x}$ and $0 \\lt \\theta \\lt 1$,\n",
     "\n",
@@ -240,24 +240,36 @@
     "& \\geq f(\\mathbf{x}) + {\\nabla f(\\mathbf{x})}^{\\text{T}} \\Delta \\mathbf{x}\n",
     "\\end{aligned}$$\n",
     "\n",
-    "2. necessity, to prove $f(\\mathbf{y}) \\geq f(\\mathbf{x}) + {\\nabla f(\\mathbf{x})}^{\\text{T}} \\Rightarrow \\nabla^{2} f(\\mathbf{x}) \\succeq 0 $\n",
+    "2. necessity, to prove $f(\\mathbf{y}) \\geq f(\\mathbf{x}) + {\\nabla f(\\mathbf{x})}^{\\text{T}} (\\mathbf{y} - \\mathbf{x}) \\Rightarrow \\nabla^{2} f(\\mathbf{x}) \\succeq 0 $\n",
+    "\n",
+    "consider tayler expansion, let $0 \\leq \\theta \\leq 1$\n",
     "\n",
     "$$\\begin{aligned}\n",
-    "\\nabla^{2} f(\\mathbf{x})\n",
-    "& = \\lim_{\\Delta \\mathbf{x} \\rightarrow 0} \\frac{\n",
-    "    f(\\mathbf{x} + \\Delta \\mathbf{x}) + f(\\mathbf{x} - \\Delta \\mathbf{x}) - 2 f(\\mathbf{x})\n",
-    "}{\n",
-    "    {\\Delta \\mathbf{x}}^{\\text{T}}{\\Delta \\mathbf{x}}\n",
-    "} \\\\\n",
-    "& \\geq \\lim_{\\Delta \\mathbf{x} \\rightarrow 0} \\frac{\n",
-    "    f(\\mathbf{x} + \\Delta \\mathbf{x}) + f(\\mathbf{x} - \\Delta \\mathbf{x}) - 2 f(\\mathbf{x})\n",
-    "}{\n",
-    "    {\\Delta \\mathbf{x}}^{\\text{T}}{\\Delta \\mathbf{x}}\n",
-    "} \\\\\n",
+    "f(\\mathbf{x} + \\Delta \\mathbf{x})\n",
+    "& = f(\\mathbf{x}) + \\nabla f(\\mathbf{x}) \\Delta \\mathbf{x}\n",
+    "+ \\frac{1}{2} {\\Delta \\mathbf{x}}^{\\text{T}} \\nabla^{2} f(\\mathbf{x} + \\theta \\Delta \\mathbf{x}) {\\Delta \\mathbf{x}}\n",
     "\\end{aligned}$$\n",
     "\n",
+    "for $f$ is convex,\n",
     "\n",
-    "\n"
+    "$$\\begin{aligned}\n",
+    "f(\\mathbf{x} + \\Delta \\mathbf{x}) \\geq f(\\mathbf{x}) + \\nabla f(\\mathbf{x}) \\Delta \\mathbf{x}\n",
+    "\\end{aligned}$$\n",
+    "\n",
+    "then\n",
+    "\n",
+    "$$\\begin{aligned}\n",
+    "{\\Delta \\mathbf{x}}^{\\text{T}} \\nabla^{2} f(\\mathbf{x} + \\theta \\Delta \\mathbf{x}) {\\Delta \\mathbf{x}}\n",
+    "& = 2 (f(\\mathbf{x} + \\Delta \\mathbf{x})\n",
+    "- f(\\mathbf{x}) - \\nabla f(\\mathbf{x}) \\Delta \\mathbf{x})\n",
+    "\\geq 0\n",
+    "\\end{aligned}$$\n",
+    "\n",
+    "for $\\Delta \\mathbf{x}$ is an arbitrary vector, according to the definition of positive semi-definite,\n",
+    "\n",
+    "$$\\begin{aligned}\n",
+    "\\nabla^{2} f(\\mathbf{x} + \\theta \\Delta \\mathbf{x}) \\succeq 0\n",
+    "\\end{aligned}$$"
    ]
   },
   {