CameraSensor NullReferenceException & Black Screen

[fearlessphoenix]

Describe the bug
Whenever using a CameraSensor Component with a game object that has an Agent script, a NullReferenceException is raised and the camera that is tied to the sensor just shows a black screen with 'No cameras rendering'. Additionally, the corresponding ML-Agent Python scripts halts when connecting.

This appears to be a duplicate of issue #5961, but that was closed without resolution.

To Reproduce
Steps to reproduce the behavior:

1. Open new project with Unity 'Universal 3D'
2. Install the ML-Agents v4.1.0 package
3. Add some game object, e.g. Cube
4. Give the game object an Agent script
5. Give the game object a CameraSensor component with default settings
6. Assign the main camera to the CameraSensor
7. Play the scene

Screenshots

Console logs / stack traces

In my main project where I was getting this problem, the console log I was getting was:

NullReferenceException: Object reference not set to an instance of an object
Unity.MLAgents.Sensors.CameraSensor.ObservationToTexture (UnityEngine.Camera obsCamera, UnityEngine.Texture2D texture2D, System.Int32 width, System.Int32 height) (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Sensors/CameraSensor.cs:159)
Unity.MLAgents.Sensors.CameraSensor.Update () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Sensors/CameraSensor.cs:115)
Unity.MLAgents.Agent.UpdateSensors () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Agent.cs:1154)
Unity.MLAgents.Agent.SendInfoToBrain () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Agent.cs:1114)
Unity.MLAgents.Agent.SendInfo () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Agent.cs:1377)
Unity.MLAgents.Academy.EnvironmentStep () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Academy.cs:586)
Unity.MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Academy.cs:43)

But in the reproduction environment described earlier, I was unable to produce any raised exception.

Environment (please complete the following information):

• Unity Version: 6000.3.6f1
• OS + version: Windows 10 & Debian 13
• ML-Agents version: ML-Agents v4.1.0
• Environment: Unity Universal 3D

NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.

[montplaisir]

Hi fearlessphoenix!

Thank you for your bug report.

To get the main camera rendering, select "Runtime Camera Enable" in the Camera Sensor. It is better to give each sensor its own dedicated camera. That is what the example scenes do.

Now, for the actual issue: it arises when two agents share a single CameraSensorComponent. That can happen when BehaviorParameters.UseChildSensors is true (default).

The PR #6324 should fix it.

[fearlessphoenix]

I Enabled 'Runtime Camera Enable' on each Camera Sensor and disabled the 'UseChildSensors' on the agent and it seemed to work so the most part but still occasionally had this issue arise while also freezing the editor.

However, I tried that pull request and that did seem to fix it.