java - XCTDaemonErrorDomain Code=41 "Not authorized for performing UI testing actions - Stack Overflow

I'm trying to automate real iOS device [iPhone] through selenium-java by physically connecting the device to MAC. Sometimes I able to run my tests but most of the times I get the below error. I'm starting Appium server by providing appium command in the terminal before running the test.

Am I missing something?

Error : Could not start a new session. Response code 500. Message: An unknown server-side error occurred while processing the command. Original error: Unable to start WebDriverAgent session. Original error: A new session could not be created. Details: Error Domain=XCTDaemonErrorDomain Code=41 "Not authorized for performing UI testing actions." UserInfo={NSLocalizedDescription=Not authorized for performing UI testing actions.}

This is the base test where I have set iOS capabilities

public static IOSDriver driver;    
public static IOSDriver getDriver()
        {
            caps.setCapability("appium:platformName","ios");
            caps.setCapability("appium:automationName", "xcuitest");
            caps.setCapability("appium:deviceName", "iPhone 15");
            caps.setCapability("appium:platformVersion", "18.3.1");
            caps.setCapability("appium:udid", "device-udid");
            caps.setCapability("appium:bundleId", "app-bundleId");
    
            
            try {
                driver = new IOSDriver(new URI("http://localhost").toURL(), caps);
            } catch (MalformedURLException | URISyntaxException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            System.out.println("Appium App configured");
            return driver;
    
        }