Make OAuth work out of the box with Apache servers (install dependencies, etc.)

[SamuAlfageme]

I was having some problems while using the OAuth app from the desktop client: owncloud/client#5668 (comment). When doing the PROPFIND with the token we got from the application on redirection:

$ curl -X PROPFIND -H "Authorization: Bearer <token>" https://<apache-server>/remote.php/webdav

... you'll find how the server is discarding the Authorization headers:

<s:message>
No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, 
No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, 
No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured
</s:message>

From some notes from previous OAuth meetings (cc/ @davivel @DeepDiver1975):

apache is eating the bearer auth headers
to fix this rewrite rules have to be set
this is already in our .htaccess
but two modules have been missing on the server: headers and rewrite
a2enmod headers
a2enmod rewrite
service apache2 restart
we need to see if there is a solution which does not require mod rewrite

Installing these modules proved to make the login from the clients work seamlessly.

EDIT @tomneedham: Apache VHost must allow our .htaccess to work by using AllowOverride All

These dependencies could be included on the app's Makefile (create one, as first step) and tackle down if there's something else missing to make the app work out-of-the-box.

[butonic]

the documentation already states that these modules should be enabled:
https://doc.owncloud.org/server/10.0/admin_manual/installation/source_installation.html#additional-apache-configurations

For ownCloud to work correctly, we need the module mod_rewrite. Enable it by running:
a2enmod rewrite
Additional recommended modules are mod_headers, mod_env, mod_dir and mod_mime:

@DeepDiver1975 @PVince81 we probably should add <apache>rewrite</apache> tag to the dependencies: https://doc.owncloud.org/server/10.0/developer_manual/app/info.html#dependencies that is only ckecked when runnign on apache. similar have eg <nginx> and whatever?

[DeepDiver1975]

Makefiles are only available at development time.
Furthermore I doubt that there is a way to check the apache config via makefile- considering permissions.

[butonic]

@DeepDiver1975 and what about checking available apache modules when installing apps?

[DeepDiver1975]

http://php.net/manual/de/function.apache-get-modules.php

but is this enough? no idea

[butonic]

would prevent installing an app if an apache module is not enabled. would fix this issue IMO

[DeepDiver1975]

would prevent installing an app if an apache module is not enabled. would fix this issue IMO

what about other web servers?

[ogoffart]

Another thing that might be worth doing is to detect this error in the client in order to advice the user properly.

[pmaier1]

@felixboehm @PVince81 please find assignment and move to 'development' milestone then

[felixboehm]

Server is fine as it is

• Documentation tells about requirements already today
• non apache servers behave completely different, ignoring now, as then custom server configs need ed anyway

Open task

• Proper error handling
  • on runtime check if bearer auth header is present, as possibly "apache is eating the bearer auth headers"
  • and if not, provide error message containing suggestion to check server configs - which can be consumed / reused by the clients

[DeepDiver1975]

its not possible to make a difference between an unauthorized request and a request where the auth header was eaten. they look identical. nothing we can do on server side from my pov

[ogoffart]

We can however detect that in the client side.
If the error message contains No 'Authorization: Bearer' header found. despite we know we sent it, we can show a specific message on the client with a link to some ressource about this error.
Is this error message translated on the server? or is it safe to just do a search for that substring?

[DeepDiver1975]

the message is not translated as it comes from the sabre dav lib

[davivel]

I don't like much the idea of matching error messages to decide logic :\

[felixboehm]

got it. So clients can handle the error gracefully, not related to server.

• Client
• iOS
• Android

[davivel]

Error message in clients will be a bit odd; something like

OAuth not really working in server. Contact your sysadmin and ask him to RTM

[ogoffart]

Actually, it seems I can't detect it in the client because Qt won't allow me to get the body of a request with a 401. (@guruz do you know a way?)

Edit: For reference, this is the patch i had in mind: https://paste.kde.org/p95vbqysr (but it does not work)

[ogoffart]

Note: is stripping that header happening in the default Apache configuration, or is it something that is specific to SolidGear's server? I was under the impression that it was a problem with the default settings, but if it is not the case, detecting this in the clients is much less important.

[DeepDiver1975]

Note: is stripping that header happening in the default Apache configuration,

yes - we have a setting in place in htacess which prevents this. but this requires mod_rewrite and htaccess handling needs to be in place

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/host-meta /public.php?service=host-meta [QSA,L]

[felixboehm]

no concept for a fix until now, moving to triage
Discussion should go on ...

[guruz]

Would it work to send an Authorization2 header instead? Maybe not worth it, needs too much client hackery.

You could add comment about the needed Apache config changes also to the app readme: https://github.com/owncloud/oauth2/blob/master/README.md
And to the app description displayed in market place: https://github.com/owncloud/oauth2/blob/master/appinfo/info.xml (or wherever market place gets its info from)

[SamuAlfageme]

We also saw this one in https://hub.docker.com/r/owncloud/server/ containers (10.0.3beta & that actually do include the required modules) - @pmaier1 pointed that OAuth app does not work there and I'm currently trying to debug what's wrong with that apache config

[SamuAlfageme]

@jvillafanez has been helping me debugging the scenario in #49 (comment) (OAuth2 app not working in https://hub.docker.com/r/owncloud/server/ 10.0.3beta1 containers) without success:

We have the required (headers & rewrite) modules loaded, the .htaccess rules from a working OAuth2-setup & almost identical dependencies/configs than some already working. But everytime after the auth. flow...

$ curl -H 'Authorization:Bearer IWRhX6brb2ePjWDW7lfggafw4LClLK3vxBFmQGnGQg9zQbCkTsUFI0yNt7IWHRzO' \
[...] \
-X PROPFIND 'https://<server>/remote.php/webdav/' \
--data-binary '<?xml version="1.0" ?>
<d:propfind xmlns:d="DAV:">
  <d:prop>
    <d:getlastmodified />
  </d:prop>
</d:propfind>
'

<s:message>
No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, 
No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, 
No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured
</s:message>

@DeepDiver1975 @butonic we might use some apache deep-knowledge here. This shit is bananas

[DeepDiver1975]

$ curl -H 'Authorization:Bearer IWRhX6brb2ePjWDW7lfggafw4LClLK3vxBFmQGnGQg9zQbCkTsUFI0yNt7IWHRzO' \

missing space after : ???

[SamuAlfageme]

@DeepDiver1975 No luck... still getting 401'd 😕

[butonic]

@SamuAlfageme paste the curl again pls. Also, are the other modules enabled? see #49 (comment)

[SamuAlfageme]

@butonic

Apache modules loaded:

docker exec owncloud apachectl -M

Loaded Modules:
 core_module (static)
 so_module (static)
 watchdog_module (static)
 http_module (static)
 log_config_module (static)
 logio_module (static)
 version_module (static)
 unixd_module (static)
 access_compat_module (shared)
 alias_module (shared)
 auth_basic_module (shared)
 authn_core_module (shared)
 authn_file_module (shared)
 authz_core_module (shared)
 authz_host_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 expires_module (shared)
 filter_module (shared)
 headers_module (shared)
 mime_module (shared)
 mpm_prefork_module (shared)
 negotiation_module (shared)
 php7_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 socache_shmcb_module (shared)
 ssl_module (shared)
 status_module (shared)

Failing PROPFIND

cURL request:

curl \
-H 'Depth:0' \
-H 'Authorization:Bearer Fqv0qmwoDmVADEopjGO5HxDaCEVAPXH4R6dAnKKjQHGcCAqJBfOgXmbjqG84KNyT' \
-H 'User-Agent:Mozilla/5.0 (Macintosh) mirall/2.4.0-nightly20170802 (build 8119)' \
-H 'Accept:*/*' \
-H 'Content-Type:text/xml; charset=utf-8' \
-H 'X-Request-ID:800a6347-df96-491c-9114-3933d47a45c0' \
-H 'Cookie:oc_sessionPassphrase=Phyp9EzdPwpqTNvAuDWyx9TXBHkFFymVzF3qiXfPVak4M5DC%2FrusFn%2BqYi%2FyJKLSml3sU8c0dig4YlXN%2B1cce4BqNJHBoRe7yIWuCo5nk21ssuhM8NSYWk6%2BRInxSKiG; ocrbn2oo82aq=lmd9o3rjmhb0i5bp9sn7kajkm5' \
-H 'Content-Length:114' \
-H 'Connection:Keep-Alive' \
-H 'Accept-Encoding:gzip, deflate' \
-H 'Accept-Language:en-US,*' \
-H 'Host:alfageme' \
-X PROPFIND 'https://alfageme/remote.php/webdav/' \
--data-binary '<?xml version="1.0" ?>
<d:propfind xmlns:d="DAV:">
  <d:prop>
    <d:getlastmodified />
  </d:prop>
</d:propfind>
'

[SamuAlfageme]

Tried enabling mod_rewrite logging as described in: http://httpd.apache.org/docs/2.4/mod/mod_rewrite.html#logging (on /etc/apache2/apache.conf) but no [rewrite: entries appear in the error.log; odd.

[SamuAlfageme]

Ok, we found it. Turns out both <Directory> directives on the virtual host definitions of our official docker images appear as <Location /> instead in the same lines on container-runtime:

$ docker run -d -p 80:80 -p 443:443 --name owncloud owncloud/server:10.0.3beta
$ diff \
    <(wget -O- -q https://raw.githubusercontent.com/owncloud-docker/base/master/rootfs/etc/apache2/sites-enabled/000-default.conf) \
    <(docker exec owncloud cat /etc/apache2/sites-enabled/000-default.conf)

8,10c8
<   Alias ${OWNCLOUD_SUB_URL} /var/www/owncloud
<
<   <Directory /var/www/owncloud>
---
>   <Location />
13c11
<   </Directory>
---
>   </Location>
28,30c26
<     Alias /owncloud /var/www/owncloud
<
<     <Directory /var/www/owncloud>
---
>     <Location />
33c29
<     </Directory>
---
>     </Location>

... this messes up with OAuth app (and potentially some others). Replacing one for the other fixes it.

Note that this switch is made in one the owncloud initialization scripts: https://github.com/owncloud-docker/base/blob/master/rootfs/etc/owncloud.d/90-services.sh#L8-L12 (since OWNCLOUD_SUB_URL is set by default to "/" toppath.conf will replace the 000-default.conf)

cc/ @tboerger any reason on why this replacement is needed (in general)? The scripts could be better documented for apache non-familiar people.

[tboerger]

I have consolidated the available configurations but currently I can't believe that this will really solve something: owncloud-docker/base@9d4cc6e

[SamuAlfageme]

@tboerger it solved the headers issues as expected:

$ docker run -d -p 80:80 -p 443:443 --name owncloud owncloud/server:10.0.3beta
4d7199b754c5fe1bae18e0045b0ee9638e2c530f61c51f125f396e5cd9616f63
$ docker exec --user www-data owncloud bash -c 'mkdir /var/www/owncloud/apps/oauth2 && curl -L https://api.github.com/repos/owncloud/oauth2/tarball/master | tar xz --strip=1 -C /var/www/owncloud/apps/oauth2 && php /var/www/owncloud/occ app:enable oauth2'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 59018  100 59018    0     0  33581      0  0:00:01  0:00:01 --:--:--  251k
oauth2 enabled

[...]

$ curl -i --insecure \
    -H 'Authorization:Bearer uW4Mwv7EV1bZ12ejZY8ZBYtfTHuXDvrzYCg81MIy7UPucBfCSQp0SDRgaU2UbWgW' \
    -H 'User-Agent:Mozilla/5.0 (Macintosh) mirall/2.4.0-nightly20170802 (build 8119)' \
    -X PROPFIND 'https://alfageme/remote.php/webdav/' \
    --data-binary '<?xml version="1.0" ?>
<d:propfind xmlns:d="DAV:">
  <d:prop>
    <d:getlastmodified />
  </d:prop>
</d:propfind>
'
HTTP/1.1 207 Multi-Status
[...]

[butonic]

@SamuAlfageme @tboerger Location directives overwrite Directory directives. See last example in http://httpd.apache.org/docs/current/sections.html#merging

Can we now close this? It is a configuration issue.

[tboerger]

It should be working fine with the containers.

[sharidas]

In my case the virtual host configuration shared below worked:

SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
Alias /testing "/home/sujith/test/owncloud/"
# for Shibboleth access
#Alias /oc-shib "/home/sujith/test/owncloud/"

<VirtualHost *:80>
<Directory /home/sujith/test/owncloud/>
  Options +FollowSymlinks
  AllowOverride All
  Satisfy Any

 <IfModule mod_dav.c>
  Dav off
 </IfModule>

 SetEnv /home/sujith/test/owncloud
 SetEnv HTTP_HOME /home/sujith/test/owncloud

</Directory>
# The change made for vim + DBGPavim
# php_value xdebug.remote_port 9009
# php_value xdebug.remote_autostart 1
</VirtualHost>

The only change I made in the virtual conf is the addition of first line:

SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1

[sharidas]

In my case the problem was with multiple virtual host configurations spread across different apache site-enabled configs. So each config file enabled in the sites-enabled had virtualhost *:80 directive. When I removed the unused ones and kept the config which I was using, it worked. Hence our configuration works out of box. Thanks @butonic for helping hand to find the real root cause. Hence closing this issue.

[butonic]

btw another cause of this might be a bad php-fpm config. fpm is not supported. that being said have a look at https://linode.com/docs/web-servers/apache/running-fastcgi-php-fpm-on-debian-7-with-apache/ how to correctly pass auth headers to it (without REDIRECT_ prefixes).