packages = ['numpy', 'matplotlib']

# Exercise 8Exploration with ergodic control

In this exercise, we will use ergodic control as an exploration mechanism for a point mass agent.

The goal is to analyze how ergodic control can be used to find a target object with an unknown location but with a prior knowledge on its location in the form of a distribution. The time took by ergodic control to find the hidden target is shown below the animation.

Ergodic control uses a prior information on the location of the target object in the form of a probability distribution. In this exercise, a mixture of Gaussians is used as distribution (represented as pink ellipsoids to represent the contours of two standard deviations).

Ergodic control computes control commands to generate a path so that the agent will spend time in the different location of the workspace in proportion to the given spatial distribution. Namely, if we would discretize the workspace and count the number of times the agent passed in each cell of this grid, the controller will ensure that over time, the resulting histogram will match the desired spatial distribution.

• Change the Gaussian parameters param.nbGaussian, param.Mu, param.Sigma and the initial position param.x0 of the agent (point in black in the animation) and observe the search behavior.
• Set the target position param.target to be a sample from the mixture of Gaussians (with equal mixing coefficients), the means param.Mu and the covariance matrices param.Sigma.
Hint: to sample from a mixture of Gaussians, you can first uniformly sample between 1 and param.nbGaussian to choose the Gaussian component to sample from.

param.x0 = np.array([.2, .3]) param.nbGaussian = 2 param.Mu = np.ones((param.nbVar,param.nbGaussian)) * .5 # Implement here param.Sigma = np.zeros((param.nbVar,param.nbVar,param.nbGaussian)) # Implement here # Sampling from GMM to define the target param.target = np.array([.5, .5]) # implement here update_ergodic_control()
param.x0 = np.array([.2, .3]) param.nbGaussian = 2 param.Mu = np.zeros((param.nbVar,param.nbGaussian)) param.Sigma = np.zeros((param.nbVar,param.nbVar,param.nbGaussian)) for i in range(param.nbGaussian): param.Mu[:,i] = np.random.uniform(0.1,0.9,param.nbVar) sigma_v = np.random.uniform(-1,1,param.nbVar) sigma_scale = np.random.uniform(0,0.1,1) sigma_regularization = np.random.uniform(0,0.01,1) sigma_v = sigma_v / np.linalg.norm(sigma_v) param.Sigma[:,:,i] = np.outer(sigma_v,sigma_v) * sigma_scale + sigma_regularization # Sampling from GMM to define the target gaussian_id = np.random.choice(np.arange(0,param.nbGaussian)) param.target = np.random.multivariate_normal(param.Mu[:,gaussian_id],param.Sigma[:,:,gaussian_id]) param.target = np.clip(param.target, 0.01, 0.99) # Target within [0,1] update_ergodic_control()