[Feature]: Dynamically Discover Appsmith Page URLs

[MichaelFalis]

Is there an existing issue for this?

• I have searched the existing issues

Summary

Hi,

Our company uses the community edition with multiple developers creating apps in both individual and shared workspaces. We each work in our own user workspace during development and then release our finished apps into shared production workspaces where users can access them. For example, if we were developing a time-tracking app to be used by the accounting department, it would be assigned to as specific programmer who would develop it in their own individual workspace and then release the completed version (by using the "fork" feature) into a departmental workspace ex: "Production-Accounting" where all of the accounting users have access.

Overall, this approach works great except there is one problem. Every time an app gets forked to the production workspace, it's URL changes. It's not a problem for departmental users because we only give them links to the URL of the Appsmith server itself so they just login and see all the workspaces and apps they have access to. However the changing URLs are a problem if we want to have one Appsmith app link directly to another Appsmith app because each time we fork a new version of an app into a production workspace, the URL changes. This essentially prevents us from creating a link from one Appsmith app to another since we'd need to manually open, inspect and re-link all link targets every time an app gets updated and re-forked to the production workspace. This issue also prevents us from dividing large projects up into smaller projects that link to each other so multiple programmers can each work on a separate part of a larger app.

To solve this problem, we need a way to dynamically discover page URLs based on the workspace name, app name and page name. All of this information is obviously stored in the internal Mongo database which could be easily queried at runtime to provide this information. The query needs to accept the following 3 fields as input parameters:

1. WorspaceName (ex: Production-Accounting)
2. AppName (ex: TimeTracker)
3. PageName (ex: Home)

and return this field as the output:
PageURL: (ex: http:///app//-)
(note this would be the deployed URL not the edit URL)

With this query, any app could dynamically obtain the URL of any other App by adding one data connection to the internal MongoDB, adding one MongoDB query, supplying the 3 input parameters and running the query to obtain the output URL. Then you could use that URL in a call to the "navigateTo" function of a button control.

Could you please supply the Mongo query to do this?

Thanks

Why should this be worked on?

Because it would provide a generic way for apps to link to each other regardless of whether they are copied and/or forked. For non-copied/non-forked apps that link to each other, it eliminates the need to program static page identifiers into URLs and provides a kind of future proofing in case they are copied and/or forked in the future. To be clear, this capability is only needed for links between separate apps since links within apps only require a page name and don't break due to copying/forking. Please see the summary above for detailed use cases.

[davd-edia]

also useful in other scenario: when you want to link to a page in a footer for example, or in any text link like <a href=>.

[Nikhil-Nandagopal]

@MichaelFalis I don't think this is the right solution to the problem. It seems you are fundamentally trying to collaborate with your developers across apps and workspaces and our git version control feature is the best way to achieve that. I'd love to know why you aren't considering using it to solve this problem because it simplifies updates to those applications and you can simply collaborate across different branches or forks of the app with a common git repo

[MichaelFalis]

Hi. I'm willing to look into git, I actually didn't realize git solved the problem of URL change that forking leads to. I guess the whole idea of hardcoding a link to another app with a fixed page id bothers me because fundamentally, I think about all pages in terms of Workspace / App / Page so it only seems natural to be able to link to a page based on those terms. In addition, I can think of a good use case where this capability would allow you to do things that would be impossible just using git. For example, lets say you want to develop a core app that has pluggable components that don't exist yet but will be created by other programmers in your company. As long as you set a naming convention so all the plugins live in the same workspace and app/page names follow a well known naming convention then the query I described above would allow your core app to dynamically discover those plugins at runtime. This means you can release the core app and give it the ability to link to plugins that didn't exist at the time you published it. Correct me if I'm wrong, but the issue with git as I understand it, is that you'd need to know each plugins' page id which can only be known after he plugin has been created. You could not dynamically discover links to plugins just by knowing the workspace and app/page naming convention because it requires inspecting the Appsmith mongo database. I suppose you could have each plugin register themselves in a database table that the core app can inspect to obtain a URL but that requires more coordination and maintenance then simply adhering to a naming convention for dynamic discovery. It's also possible that the plugin may use a different data source from the core app because the developer is unfamiliar with it - in that case there wouldn't be a common table. Overall, I just not a fan of statically coding something that can be dynamically discovered at runtime so easily. Dynamic link discovery also gives your app more flexibility to integrate with other apps that may not even exist yet and eliminates the need to hard-code a plugin link and re-publish your core app just because a new plugin was developed at a later point in time.

(note: the term "plugin" used above just means another appsmith app that the core app wants to link to or communicate with via query url parameters)

[MichaelFalis]

Here's one more use case. Let's say your company has many different departments and you don't want to clutter your appsmith workspaces with a separate appsmith workspace for each department (as I described in my first post), you may want to build a generic app launcher. This could be useful in a medium to large corporate environment where you want to permission users by department. The launcher would maintain an internal table that associates each user with a department. All production apps would need to adhere to the following standards:

1. They must exist in the "Production" workspace
2. The app name must always begin with the department name that should have access and be followed by an underscore.
   ex: Accounting_Timetracker
3. The primary page of the app must always be named "Home"

Using this simple setup, along with the query I described above, it would be easy to create a generic app launcher that only shows users apps they have permissions to (based on their department) and would not need to be re-published every time a new app is added to the Production workspace. It wouldn't require any special knowledge of the apps it's launching and wouldn't need to share any common data tables either.

[finchett]

Curious about this as well. I want to embed appmith in an iframe but I've noticed that page URLs change when I deploy in another appsmith instance.

https://localhost/app/application-name/some-page-{changing part}

Is this {changing part} supposed to persist if using git integration? I haven't found that to be the case so far.

[MichaelFalis]

@finchett - Exactly, the query I described above would also work with your use case of multiple instances. All you'd need to do is point the query to the other instance's internal mongo database and you could dynamically obtain the url at runtime. Unfortunately, I haven't yet figured out how to access the internal mongodb from a different machine on the local network. I believe it involves making changes to the yml file and the mongo config files. If the appsmith team could explain how to do this, it would be extremely helpful.

Another benefit of being able to connect to the internal appsmith database remotely is that it would allow me to connect via our postgres data warehouse via the mongo foreign data wrapper. This would allow me to put the mongo query in the data warehouse that is used by all of our developers and they they could all dynamically obtain page urls without needing to explicitly add the mongo connection and query into every app they create. Instead they'd just call a single postgres function, supply the parameters and receive the the url they need as a return value.

I understand there are security concerns with remotely accessing the internal mongo database but our network is behind a firewall and our apps are only used by local users so in this situation, the benefits far outweigh the risks.

[davd-edia]

IMHO having a new appsmith property with an array of pages URLs would be the simplest approach.

[MichaelFalis]

@davd-edia Agreed - I don't know how long that would take, but with the correct query we would have the functionality now. Everything needed already exists in the database.

[MichaelFalis]

I figured out how to connect to the internal appsmith database remotely. If anyone is interested, see the instructions at the bottom of this post:

https://discord.com/channels/725602949748752515/1239099101963091988

Now, all I need is the query requested above.

[MichaelFalis]

I figured out the needed mongo query and posted it below. Here are the steps to create a demo project:

1. Create a new application and create new data connection to mongodb.
2. Use the settings shown in step 9 in above discord link. You can just use "localhost" instead of IP.
3. Create a new query named "qryParam" with these settings:
   Command: Aggregate
   Collection: workspace
   Array of pipelines:

[
{$addFields: { "wsid": { "$toString": "$_id" } } }
, {
$lookup:{
"from":"application",
"localField":"wsid",
"foreignField":"workspaceId",
"as":"app"
}
},
{ $unwind : '$app'},
{
$project:{
_id:0
,ws_email:"$email"
,ws_name:"$name"
,ws_deleted:"$deleted"
,ws_id:"$wsid"
,ws_slug:"$slug"
,app__id:"$app._id"
,app_name:"$app.name"
,app_slug:"$app.slug"
,app_deleted:"$app.deleted"
}
},
{ $addFields: { "app_id": { "$toString": "$app__id" } } },
{
$lookup:{
"from":"newPage",
"localField":"app_id",
"foreignField":"applicationId",
"as":"page"
}
},
{ $unwind : '$page'},
{ $match:
{
"ws_deleted":false
,"app_deleted":false
,"page.deleted":false
,"page.deletedAt":{$exists:false}
}
},
{ $addFields: { "page_id": { "$toString": "$page._id" } } },
{
$project:{
_id:0
,ws_name:"$ws_name"
,ws_id:"$ws_id"
,ws_slug:"$ws_slug"
,app_id:"$app_id"
,app_name:"$app_name"
,app_slug:"$app_slug"
,pg_pub_name:"$page.publishedPage.name"
,pg_slug:"$page.publishedPage.slug"
,pg_id:"$page_id"
,pg_pub_link:{ "$concat": [ "/app/", "$app_slug","/", "$page.publishedPage.slug","-","$page_id"] },
}
},
{
$sort:{ws_name:1,app_name:1,pg_pub_name:1,pg_unpub_name:1}
},
{ $match:
{
"ws_name":{$regex: ".{{Input1.text}}.",$options:"i"},
"app_name":{$regex: ".{{Input2.text}}.",$options:"i"},
"pg_pub_name":{$regex: ".{{Input3.text}}.",$options:"i"}
}
}
]

 Limit: 1000

4. Drag 4 input controls and one button to the form and configure like this:

Input1:
name: "workspace"
ontextchanged:

{{qryParam.run({
"workspace_name": Input1.text,
"application_name": Input2.text,
"page_name": Input3.text
});}}

Input2:
name: application
ontextchanged:

{{qryParam.run({
"workspace_name": Input1.text,
"application_name": Input2.text,
"page_name": Input3.text
});}}

Input3:
name: page
ontextchanged:

{{qryParam.run({
"workspace_name": Input1.text,
"application_name": Input2.text,
"page_name": Input3.text
});}}

Input4:
text: Published Page Link
default value:
{{qryParam.data[0].pg_pub_link}}

Button1:
label: Launch Page
onClick:
{{navigateTo('http://{appsmith server IP}' + qryParam.data[0].pg_pub_link, {}, 'NEW_WINDOW');}}

Be sure to replace {appsmith server IP} with your server's IP address.
Also, turn off "Smart BSON Substitution" under query settings.