AdaSwarm: A Novel PSO optimization Method for the Mathematical Equivalence of Error Gradients

This paper tackles the age-old question of derivative free optimization in neural networks. This paper introduces AdaSwarm, a novel derivative-free optimizer to have similar or better performance to Adam but without "gradients". To support the AdaSwarm, a novel Particle Swarm Optimization Exponentially weighted Momentum PSO (EM-PSO), a derivative-free optimizer, is also proposed which tackles constrained and unconstrained single objective optimization problems and looks at applying the proposed momentum particle swarm optimization on benchmark test functions, engineering optimization problems and habitability scores for exoplanets which show speed and convergence of the technique. The EM-PSO is extended by approximating the gradient of a function at any point using the parameters of the particle swarm optimization. This is a novel technique to simulate gradient descent, an extremely popular method in the back-propagation algorithm, using the approximated gradients from the particle swarm optimization parameters. Mathematical proofs of gradient approximation by EM-PSO, thereby bypassing the gradient computation, are presented. The AdaSwarm is compared with various optimizers and the theory and algorithmic performance are supported by promising results.

A Framework for Democratizing AI

Machine Learning and Artificial Intelligence are considered an integral part of the Fourth Industrial Revolution. Their impact, and far-reaching consequences, while acknowledged, are yet to be comprehended. These technologies are very specialized, and few organizations and select highly trained professionals have the wherewithal, in terms of money, manpower, and might, to chart the future. However, concentration of power can lead to marginalization, causing severe inequalities. Regulatory agencies and governments across the globe are creating national policies, and laws around these technologies to protect the rights of the digital citizens, as well as to empower them. Even private, not-for-profit organizations are also contributing to democratizing the technologies by making them \emph{accessible} and \emph{affordable}. However, accessibility and affordability are all but a few of the facets of democratizing the field. Others include, but not limited to, \emph{portability}, \emph{explainability}, \emph{credibility}, \emph{fairness}, among others. As one can imagine, democratizing AI is a multi-faceted problem, and it requires advancements in science, technology and policy. At \texttt{mlsquare}, we are developing scientific tools in this space. Specifically, we introduce an opinionated, extensible, \texttt{Python} framework that provides a single point of interface to a variety of solutions in each of the categories mentioned above. We present the design details, APIs of the framework, reference implementations, road map for development, and guidelines for contributions.