BI+ distributed setup "startup document does not exist " "GSM unable to acquire LSM"

We have a network deployment for Foundation and BI+ in our environment and the 2nd node when operated on its own stopped being accessible. RAF agent service kept going to "stopping" mode when stopped and not completely stopped. The RAF Agent service and the Foundation Service was set to the installer account.

I thought it will be a good idea to compile a few things I verified in the hopes that it will help anyone with some troubleshooting ideas.

Ensure to set HyS9RaFrameworkAgent_epmsystem1 service to start with Domain account instead of Local System on both RAF servers and also make sure the domain account has local administrator rights .

Before starting services on Server2 make sure server1 is up and running and Workspace is accessible

Verify that HAVA ID in EPM registry matched in css_provisioning_info table

1: Execute the following query against the Shared Services registry database:
2: select application_id from css_provisioning_info where lower(application_id) like 'hava%';
o note this application id, e.g. HAVA:0000012b3072bf63-0000-6717-0acc1bac

3: Run epmsys_registry.bat from \Oracle\Middleware\user_projects\epmsystem1\bin
Open EPM registry.xml and search for HAVA and compare the HAVA ID using the ID above

4: If there is a mismatch, edit the Shared Services Registry:
Go to EPM_ORACLE_INSTANCE/bin and run the following command:
epmsys_registry updateproperty RA_FRAMEWORK/@applicationId

Where applicationId is the id returned from the query above. For example, using the id above:

epmsys_registry updateproperty RA_FRAMEWORK/@applicationId HAVA: 0000012b3072bf63-0000-6717-0acc1bac

5. Restart all services on Server1 and ensure Shared Services and Workspace are accessible

6: Enable the Service Broker service and the GSM on the second server
- Log on to EPM Workspace on Server1
- Select Navigate, then Reporting and Analysis, and then Services.
- Right-click Reporting and Analysis for server2 , and then select Properties.
- On the Services tab, set GSM to Enabled (Yes) and Service Broker to Enable (Yes).

Reproduce the issue and attach the logs from these locations from both servers 1/2
- \Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis

and run epmsys_registry.bat from \Oracle\Middleware\user_projects\epmsystem1\bin
For my issue however, everything seemed to tie up quite nicely and I was still running into the issue.
This was the log entry from the server 2  raframework_stdout_console.log when started on its own:

Finally, when I started digging a little deeper, I could understand what was causing this to happen.
While Configuring the RAF for the 2nd node, We normally point to the shared server 1 RM1 directory while configuring.

The second Workspace agent was not able to locate the RM1 directory on server 1 .An issue with DNS could not resolve the server 1 hostname correctly.On second Workspace Agent server, add the server 1 hostname/IP address into the HOSTS file

On server 2, update the hosts file (which is under C:/Windows/System32/etc/drivers/hosts folder) with the ipaddress and the hostname of server1 where RM1 directory stored.
xx.xx.xx.xx server1.domain.com

This helped in resolving the issue.

.