Dear Anandkumar,
Good day! Thank you for posting to Microsoft Community. We are happy to help you.
Based on your description, it seems you created a list on Microsoft list using a Power App custom form, but when you click "New" button to add some records, it opens the last visited page! We apologize for the inconveniences caused!
We would also love to help you on this question, but our support category handles general SharePoint questions. Now that your question involves an issue with custom list created using PowerApps, we may not have resources to handle issues related to PowerApps.. Therefore, to avoid giving inconsistent information and to make sure you get expert help about your question, we suggest you post a new thread in Power Apps Community. This is a specific and dedicated channel for the issues and questions related to your scenario. The members and moderators in this community are specialized to find the related answer to your concern. I am sure our Expert Engineers from that side will effectively and accurately address your issue.
Thank you for your kind understanding and cooperation. Hope you have a great day and stay safe!
With Sincerest Regards,,
Edwine | Microsoft Community Moderator