{"id":78561,"date":"2020-09-14T09:00:25","date_gmt":"2020-09-14T03:30:25","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=78561"},"modified":"2021-05-09T13:13:22","modified_gmt":"2021-05-09T07:43:22","slug":"scipy-optimization","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/","title":{"rendered":"SciPy Optimization &#8211; Unconstrained, Constrained, Least- Square, Univariate Minimization"},"content":{"rendered":"<p>SciPy consists of an optimization module. Optimization is the final step in any technological process. It is one of the most important packages. We optimize the input parameters of a function. The module scipy.optimize() consists of a number of different optimization algorithms.<\/p>\n<h2>Optimization in SciPy<\/h2>\n<p>We can optimize the parameters of a function using the <strong>scipy.optimize()<\/strong> module. It contains a variety of methods to deal with different types of functions.<\/p>\n<p><strong>1. minimize_scalar()-<\/strong> we use this method for single variable function minimization.<br \/>\n<strong>2. minimize()-<\/strong> we use this method for multivariable function minimization.<br \/>\n<strong>3. curve_fit()-<\/strong> We use this method for fixing a function to a data set.<br \/>\n<strong>4. root_scalar()-<\/strong> It is to determine the zeros of a single variable function.<br \/>\n<strong>5. root()-<\/strong> It is to determine the zeros of multivariable function<br \/>\n<strong>6. linprog()-<\/strong> We use it to minimize a linear function with equality and inequality properties.<\/p>\n<p>We can perform various types of minimizations:<\/p>\n<ul>\n<li>Constrained and unconstrained minimization of single or multi-variable scalar functions. We can perform this using various minimization algorithms<\/li>\n<li>Global optimization.<\/li>\n<li>Least- square minimizations and curve fittings.<\/li>\n<li>For Root finding.<\/li>\n<li>Solving multivariate equation systems.<\/li>\n<\/ul>\n<h2>Unconstrained and Constrained Minimization in SciPy<\/h2>\n<p>We use the minimize()function for the performing minimization on the scalar function. As an example function, we use the Rosenbrock scalar function.<\/p>\n<p><strong>f(x,y)=(1\u2212x)2+100(y\u2212x2)2<\/strong><\/p>\n<p>The minimum value is 0. We can achieve this by setting x=1.<\/p>\n<p>We can perform this process using two methods:<\/p>\n<h3>1. Nelder-Mead Simplex Algorithm in SciPy<\/h3>\n<p>We use the minimize function to set the method as \u2018Nedler-Mead\u2019. This is one of the easiest techniques. However, it is a very slow process.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"null\">import numpy as np\r\nfrom scipy.optimize import minimize\r\ndef rosen(x):\r\n  return sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0)\r\n \r\nx0 = np.array([1.9, 0.7, 3.4, 1.6, 2.1])\r\nres = minimize(rosen, x0, method='nelder-mead')\r\nprint(res.x)\r\n<\/pre>\n<p><strong>Output<\/strong><\/p>\n<div class=\"code-output\">[1.00000028 0.99999778 0.99999712 0.9999944 0.99998762]<\/div>\n<h3>2. Powell Algorithm in SciPy<\/h3>\n<p>This is a similar method. It uses a single scalar function call. We can implement it using the method as Powell.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"null\">import numpy as np\r\nfrom scipy.optimize import minimize\r\ndef rosen(x):\r\n  return sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0)\r\n \r\nx0 = np.array([1.9, 0.7, 3.4, 1.6, 2.1])\r\nres = minimize(rosen, x0, method='powell')\r\nprint(res.x)\r\n<\/pre>\n<p><strong>Output<\/strong><\/p>\n<div class=\"code-output\">[1. 1. 1. 1. 1.]<\/div>\n<h2>SciPy Least- Square Minimization<\/h2>\n<p>We can solve the least squares with bound variables. We take two inputs &#8211; a residual cost function and a loss scalar function and calculate the minimum cost function.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"null\">import numpy as np\r\nfrom scipy.optimize import minimize\r\nfrom scipy.optimize import least_squares\r\ndef rosen(x):\r\n  return sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0)\r\n \r\ninput = np.array([1, 3])\r\nres = least_squares(rosen, input)\r\n \r\nprint (res)\r\n<\/pre>\n<p><strong>Output<\/strong><\/p>\n<div class=\"code-output\">active_mask: array([0., 0.])<br \/>\ncost: 0.07163696298945274<br \/>\nfun: array([0.37851542])<br \/>\ngrad: array([0.1312017 , 0.10353955])<br \/>\njac: array([[0.3466218 , 0.27354117]])<br \/>\nmessage: &#8216;The maximum number of function evaluations is exceeded.&#8217;<br \/>\nnfev: 200<br \/>\nnjev: 194<br \/>\noptimality: 0.1312016979177483<br \/>\nstatus: 0<br \/>\nsuccess: False<br \/>\nx: array([1.61508403, 2.60986412])<\/div>\n<h3>1. Root Finding<\/h3>\n<p>We can perform root-finding on scalar equations. There are 4 different algorithms for root- finding. We can evaluate an equation through its endpoints. We consider the root of the endpoint as the function changes sign. The most efficient technique is \u2018brenqt\u2019.<\/p>\n<h3>2. Fixed-Point solving<\/h3>\n<p>We can find the zeroes of single or multivariate equations. The problem is equivalent to finding a fixed point of the function. We can use the Aitkens sequence acceleration technique. It is a simple and iterative method.<\/p>\n<h3>3. Equation system<\/h3>\n<p>We can find the root of a set of nonlinear equations. There are several methods available for this purpose.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"null\">import numpy as np\r\nfrom scipy.optimize import root\r\ndef func(x):\r\n   return x*2 + 2 * np.sin(x)\r\nsol = root(func, 0.3)\r\nprint(sol)\r\n<\/pre>\n<p><strong>Output<\/strong><\/p>\n<div class=\"code-output\">fjac: array([[-1.]])<br \/>\nfun: array([0.])<br \/>\nmessage: &#8216;The solution converged.&#8217;<br \/>\nnfev: 7<br \/>\nqtf: array([-2.21298214e-20])<br \/>\nr: array([-4.])<br \/>\nstatus: 1<br \/>\nsuccess: True<br \/>\nx: array([0.])<\/div>\n<h2>SciPy Univariate Minimization<\/h2>\n<p>Similar to multivariate we can perform for single variables. We use the minimize_scalar function in this case. We use the \u2018brent\u2019 method for unconstrained minimization. For bounded univariate minimizations, we use the \u2018bounded\u2019 method.<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"null\">from scipy.optimize import minimize_scalar\r\nf = lambda x: (x - 2) * (x + 1)**2\r\nres = minimize_scalar(f, method='brent')\r\nprint(res.x)\r\n<\/pre>\n<p><strong>Output<\/strong><\/p>\n<div class=\"code-output\">1.0<\/div>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"null\">from scipy.special import j1\r\nres = minimize_scalar(j1, bounds=(4, 7), method='bounded')\r\nres.x\r\n \r\n<\/pre>\n<p><strong>Output<\/strong><\/p>\n<div class=\"code-output\">5.3314418424098315<\/div>\n<h2>Summary<\/h2>\n<p>Optimization is the most important concept for any function evaluation. We have the optimization module in SciPy. It is very convenient for working with scalar functions. It consists of a wide range of methods and algorithms. We can deal with linear, nonlinear, multivariate, and univariate equations.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>SciPy consists of an optimization module. Optimization is the final step in any technological process. It is one of the most important packages. We optimize the input parameters of a function. The module scipy.optimize()&#46;&#46;&#46;<\/p>\n","protected":false},"author":6,"featured_media":82104,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22401],"tags":[23254,23253,23251,23252],"class_list":["post-78561","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-numpy","tag-minimizations-in-scipy","tag-optimization-in-scipy","tag-scipy-optimization","tag-scipy-optimize"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>SciPy Optimization - Unconstrained, Constrained, Least- Square, Univariate Minimization - DataFlair<\/title>\n<meta name=\"description\" content=\"SciPy optimization provides function for minimizing objective function subject to constraints.Unconstrained,constrained,Least Square,Univariate minimization\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/scipy-optimization\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"SciPy Optimization - Unconstrained, Constrained, Least- Square, Univariate Minimization - DataFlair\" \/>\n<meta property=\"og:description\" content=\"SciPy optimization provides function for minimizing objective function subject to constraints.Unconstrained,constrained,Least Square,Univariate minimization\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/scipy-optimization\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-09-14T03:30:25+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-05-09T07:43:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Optimization-in-SciPy-df.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"SciPy Optimization - Unconstrained, Constrained, Least- Square, Univariate Minimization - DataFlair","description":"SciPy optimization provides function for minimizing objective function subject to constraints.Unconstrained,constrained,Least Square,Univariate minimization","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/","og_locale":"en_US","og_type":"article","og_title":"SciPy Optimization - Unconstrained, Constrained, Least- Square, Univariate Minimization - DataFlair","og_description":"SciPy optimization provides function for minimizing objective function subject to constraints.Unconstrained,constrained,Least Square,Univariate minimization","og_url":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2020-09-14T03:30:25+00:00","article_modified_time":"2021-05-09T07:43:22+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Optimization-in-SciPy-df.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89"},"headline":"SciPy Optimization &#8211; Unconstrained, Constrained, Least- Square, Univariate Minimization","datePublished":"2020-09-14T03:30:25+00:00","dateModified":"2021-05-09T07:43:22+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/"},"wordCount":582,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Optimization-in-SciPy-df.jpg","keywords":["Minimizations in scipy","Optimization in scipy","SciPy Optimization","SciPy Optimize"],"articleSection":["NumPy Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/scipy-optimization\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/","url":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/","name":"SciPy Optimization - Unconstrained, Constrained, Least- Square, Univariate Minimization - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Optimization-in-SciPy-df.jpg","datePublished":"2020-09-14T03:30:25+00:00","dateModified":"2021-05-09T07:43:22+00:00","description":"SciPy optimization provides function for minimizing objective function subject to constraints.Unconstrained,constrained,Least Square,Univariate minimization","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/scipy-optimization\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Optimization-in-SciPy-df.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2020\/09\/Optimization-in-SciPy-df.jpg","width":1200,"height":628,"caption":"Optimization in SciPy"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/scipy-optimization\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"NumPy Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/numpy\/"},{"@type":"ListItem","position":3,"name":"SciPy Optimization &#8211; Unconstrained, Constrained, Least- Square, Univariate Minimization"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/2c58ecb4f73a39f0ef993f1ddfcd7b89","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ce4a0e3e542444fc73bbebf83e89e8b73e2d95ccb1fcee64da9945f078b97c5?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"The DataFlair Team provides industry-driven content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our expert educators focus on delivering value-packed, easy-to-follow resources for tech enthusiasts and professionals.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam2\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/78561","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=78561"}],"version-history":[{"count":4,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/78561\/revisions"}],"predecessor-version":[{"id":92986,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/78561\/revisions\/92986"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/82104"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=78561"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=78561"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=78561"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}