Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drupal Console + Aegir #2263

Open
mstenta opened this issue May 15, 2016 · 37 comments
Open

Drupal Console + Aegir #2263

mstenta opened this issue May 15, 2016 · 37 comments

Comments

@mstenta
Copy link

mstenta commented May 15, 2016

Drupal Console does not seem to be working with sites installed via Aegir. I think there may be two separate issues that are preventing it from working.

  1. Aegir uses Drupal multisite for all sites that it installs.

Based on the Console documentation, and #755, it seems like Console does support multisite installations, but it also seems that it is not currently working. I believe part of the issue is #2002

  1. Aegir does not store database credentials in settings.php.

As an extra security measure, Aegir stores database credentials in the Apache virtual host that it creates, and loads them dynamically into settings.php temporarily. Once the database connection is established, it clears the credential variables so that they cannot be accessed by anyone with PHP access (or phpinfo()) within the site itself.

In order for Drush to work, Aegir also stores the database credentials in sites/[site-uri]/drushrc.php.

Perhaps Console could also look in drushrc.php for credentials, if it doesn't find them in settings.php? This could be done conditionally, so that we don't have to depend on Drush of course.

@colans
Copy link

colans commented Jun 5, 2017

@jmolivas Would you be able to at least point us to the code location where we could add a patch for this, for number 2 above? Thanks.

@jmolivas
Copy link
Member

jmolivas commented Jun 5, 2017

@colans What will be the easies way to test this myself? I mean to have an Aegir setup locally?

@colans
Copy link

colans commented Jun 5, 2017

Folks traditionally use Valkyrie, but I use a real Dev VM hosted somewhere.

@ergonlogic: Is there a newer/better way nowadays?

@ergonlogic
Copy link

Valkyrie should work, but also adds a number of components specific to local development. An alternative would be to launch a Vagrant VM from https://github.com/ergonlogic/ansible-role-aegir, which should be much closer to a stock Aegir install. There are also some Docker-based deployment options, but I'm not familiar enough with them to suggest whether they'd be helpful here.

@ttadegraaff
Copy link

ttadegraaff commented Aug 30, 2017

It looks as if only problem nr. 1 is an issue. If I add a symbolic link sites/default/settings.php pointing to the sites/my-site.org/settings.php then drupal console seems to be functional with the database from my-site.org.

@esolitos
Copy link
Contributor

Yes that works, and if you have 1 site per platform it's ok, but if you have more (which is the case, usually), this is not a solution.

@ttadegraaff
Copy link

ttadegraaff commented Aug 30, 2017

Well I didn't say it's a solution. I only specified that only problem nr. 1 mentioned in the first post seems to be an issue, and thus problem nr. 2 not. ;)

This seems to me to be important information for someone willing to fix the problem mentioned in this thread. It's probably not an aegir problem, so probably no need to create Aegir instances for testing purposes etc. There might already be another issue pointing to the lack of drupal console support for multisite, so this issue might be closed...?

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

@ttadegraaff What Console commands did you test? Can you test one that interacts with the Drupal database?

If you tested one that does not require a connection to the Drupal database, then it did not test whether or not the 2nd problem I described is still an issue. The second problem I described is that Aegir does not put database credentials inside settings.php - it puts them inside the Apache virtual host - which means that Console will only be able to get the database credentials if it is able to get those from the virtual host as well (running in the context of an Apache request).

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

PS: There is already an open issue about Console not working with multisite: #2002

If it is true that a symlinked settings.php works with the database, then that is good news - it means we can close this issue and focus on #2002 - because like you said it is not an Aegir issue.

@ttadegraaff
Copy link

ttadegraaff commented Aug 30, 2017

I did a

drupal debug:user

with result:

// Welcome to the Drupal user debug


User ID Username Roles Status


1 spadmin Authenticated user, Administrator Enabled


@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

@ttadegraaff Ok that's great! Thanks for confirming!

I think that confirms that this is not an Aegir issue, and we can focus efforts on #2002 instead. Does everyone agree?

I am curious about HOW Console is getting the database credentials when they are not in settings.php... I don't know the inner workings of Console enough, though. Does it perform an Apache request that then loads up the virtual host information?

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

As I stated in the issue description:

In order for Drush to work, Aegir also stores the database credentials in sites/[site-uri]/drushrc.php.

Perhaps Console could also look in drushrc.php for credentials, if it doesn't find them in settings.php? > This could be done conditionally, so that we don't have to depend on Drush of course.

Is Console looking in drushrc.php? Will this still work if drushrc.php is not present?

@ttadegraaff
Copy link

Curious... I've renamed drushrc.php to drushrc.php.backup, and drupal console is still functional, same output as above. What's even more interesting, and troubling... drush also continues functional, for example a cache-rebuild.

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

VERY interesting. Does your settings.php have database credentials in it?

@ttadegraaff
Copy link

Oooopss........ My bad... I'd hard coded database credentials for testing purpose before....

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

Ah ok - good thing we checked! So both 1 and 2 might still be issues here. Thanks for helping to investigate @ttadegraaff .

@ttadegraaff
Copy link

They are, I'm sorry for the misinformation.

@jmolivas
Copy link
Member

@mstenta @ttadegraaff we are more than happy to help, mentioning @omero since he has more experience with DevOps tasks.

Could any of you do a screen-share call to show us the issues and have a better understanding since we are not very related to Aegir project.

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

Thanks @jmolivas ! It probably makes sense to focus on #2002 first, since Aegir uses multisite for all Drupal installations - so Console needs to be able to work with multisite as a prerequisite for this issue.

The other piece (database credentials not in settings.php) might be trickier... I'd be happy to look at that together after #2002 is fixed.

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

Oh actually... maybe #2002 is fixed already? I see that the latest comment points to this merged pull request: #2850

@jmolivas
Copy link
Member

@mstenta DrupalConsole supports multi-sites already

@jmolivas
Copy link
Member

You can try by using drupal --uri=MULTISITE_NAME COMMAND

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

@jmolivas Yea, but there was a bug that prevented it from working for a while. That might have been fixed though? I just commented on #2002 to ask @mickaelperrin if it was. He made pull request #2850 which seems to be related to it.

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

If it is fixed, that's great! That means the only thing we need to focus on in this issue is the second thing I described in this issue's original description:

  1. Aegir does not store database credentials in settings.php.

As an extra security measure, Aegir stores database credentials in the Apache virtual host that it creates, and loads them dynamically into settings.php temporarily. Once the database connection is established, it clears the credential variables so that they cannot be accessed by anyone with PHP access (or phpinfo()) within the site itself.

In order for Drush to work, Aegir also stores the database credentials in sites/[site-uri]/drushrc.php.

Perhaps Console could also look in drushrc.php for credentials, if it doesn't find them in settings.php? This could be done conditionally, so that we don't have to depend on Drush of course.

I'm not sure if there's any easy solution to that... Console relies on the fact that settings.php contains the database credentials. With Aegir, the database credentials are stored in the Apache virtual host file.

@mstenta
Copy link
Author

mstenta commented Aug 30, 2017

As I said, one option would be for Console to conditionally look for database credentials in sites/[uri]/drushrc.php, but that feels messy because it would be very specific to this Aegir issue.

If we want Console to work with Aegir, I think there are only two options:

  1. Console adds a little code that is specifically for dealing with this particular Aegir detail.

or

  1. Aegir starts putting database credentials in settings.php

So: does Aegir compromise for Console? Or does Console compromise for Aegir?

@colans
Copy link

colans commented Aug 30, 2017

Let's forget about having Console look into Web server config files. This gets messy really fast as both Apache and Nginx are supported.

I'm throwing my weight behind having Console look into drushrc.php. I believe this to be the simplest solution that doesn't compromise Aegir's security model.

So if Console doesn't find credentials in settings.php, it should look for a drushrc.php file. If it doesn't exist or is missing credentials, then fail. I don't believe this to be too much of a compromise given that we're simply adding support for the one and only open-source Drupal hosting system.

It shouldn't be too hard to produce a patch for this as all we need to know is where in the Console code those credentials are dug up.

Just to state my bias though, I'm a core Aegir maintainer. ;)

@ttadegraaff
Copy link

ttadegraaff commented Aug 30, 2017

To make up for the disinfo I've spreaded this afternoon ;) I've hacked a little dirty bash script that may be useful for some people.

Disclaimer, I hardly ever write bash scripts, so dirty probably means really dirty ;)

Put mydrupal in /usr/local/bin and set executable. Now change into a site directory, and use mydrupal with the aegir user, and with the same arguments as you would normally use drupal.

How it works? When executing mydrupal a sites/default/settings.php is created which includes the settings.php from the site and with the site's database settings from drushrc.php. After that the drupal command is executed with the same arguments as given to the mydrupal command. As the drupal command uses the database credentials from sites/default/settings.php you get the correct results. After executing the drupal command the sites/default/settings.php is cleaned up again.

See for mydrupal code:
https://pastebin.com/Ej0rScNt

@ttadegraaff
Copy link

@jmolivas I'd be happy to be of any use in solving this issue by providing info, but I do not know what a 'screen-share call' means? (mentioned in #issuecomment-326028044)

@jmolivas
Copy link
Member

jmolivas commented Aug 30, 2017

@ttadegraaff A hangout/skype/gtm call where any of any of you can share your screen from your local where Aegir is running.

@jonpugh
Copy link

jonpugh commented Aug 31, 2017

@jmolivas Try the aegir development toolkit: https://github.com/aegir-project/development/

Install Docker, docker-compose, and drush locally then launch that sucker.

@ttadegraaff
Copy link

Hi, I've created a fix that seems to work. I've created a local branch from 1.0.1, named it 2263-aegir-support, commited some changes, tested them, it seems to work, and that's where I'm now. I found some information about creating a pull request, but I can't push the branch. Can anybody help me helping fixing this issue? :)

I would appreciate a code review with some feedback. I'm a beginner in object oriented programming (and Drupal 8), so my skills are quite limited. The database settings in Drupal Console are created in Drupal\Core\Site\Settings which is a final class. As I understand this makes it impossible to extend the class, so I copied and renamed the class in Drupal Console, and made my changes. This is quite ugly, I hope there is a nicer way to do this.

@jonpugh
Copy link

jonpugh commented Sep 2, 2017

@ttadegraaff You should submit a pull request for your branch so we can all easily take a look and suggest changes.

Where is your branch? I'd love to see it.

@jonpugh
Copy link

jonpugh commented Sep 2, 2017

@jmolivas I'd be happy to sprint with you on this, using an aegir on my machine or perhaps I can help you get it running on yours.

Any time next week work for you?

@ttadegraaff
Copy link

My branch is local. I want to push it, but I get an error message:

aegir@aeg.loc:~/platforms/drupal-8.3.1/vendor/drupal/drupal-console$ git push origin 2263-aegir-support Username for 'https://github.com': ttadegraaff Password for 'https://ttadegraaff@github.com': remote: Permission to hechoendrupal/drupal-console.git denied to ttadegraaff. fatal: unable to access 'https://github.com/hechoendrupal/drupal-console.git/': The requested URL returned error: 403

@jonpugh
Copy link

jonpugh commented Sep 2, 2017

At the top of this page is a button "Fork". Click that and you will get a copy of this repository you can push to.

Change the git remote URL to your Fork by editing .git/config file and you should be able to push to that one.

Then, go to the pull requests section of this repository to submit it...

@ttadegraaff
Copy link

Thx! :)

The pull request is created:
#3509

@ttadegraaff
Copy link

@jonpugh I see the pull request is still open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants