Devstack plugin to federate with testshib.org

In a previous patch, I implemented a Devstack plugin to enable
federation and idp features in keystone. The plugin was to be
configured from environment variables for the idp entityID, metadata,
sp_auth_url, sp_url, etc. Providing an endless and untestable matrix
of combinations. Therefore the review was gathering dust waiting for
brave reviewers.

This review extracts the meat of the previous patch and removes all
the configuration options. This plugin now does one thing only: It
installs mod_shibboleth and sets up testshib.org as the IdP for keystone.

While testshib.org will not be used in our functional testing, this
is a necessary first step to make such complex changes more testable
reproducible and reviewable.

A follow-up patch will install a shibboleth-idp, and either that one,
or a later one, will switch from testshib.org to the local shibboleth.

This plugin will not yet be run as part of the gate, as "enable_service
federation" needs to be added to the Devstack options.

To run add the following after the lines that set up keystone from a
gerrit review:

enable_plugin keystone $KEYSTONE_REPO
enable_service keystone-saml2-federation

Change-Id: I6f7491ff063359d7065c77b00fe5bfc76f8587d6

5 files changed, 183 insertions, 1 deletions

diff --git a/devstack/files/federation/shib_apache_alias.txt b/devstack/files/federation/shib_apache_alias.txt
new file mode 100644
index 000000000..8453b5939
--- /dev/null
+++ b/devstack/files/federation/shib_apache_alias.txt
@@ -0,0 +1 @@
+    WSGIScriptAliasMatch ^(/v3/OS-FEDERATION/identity_providers/.*?/protocols/.*?/auth)$ /var/www/keystone/main/$1
diff --git a/devstack/files/federation/shib_apache_handler.txt b/devstack/files/federation/shib_apache_handler.txt
new file mode 100644
index 000000000..117e022dc
--- /dev/null
+++ b/devstack/files/federation/shib_apache_handler.txt
@@ -0,0 +1,16 @@
+
+<Location /Shibboleth.sso>
+    SetHandler shib
+</Location>
+
+<Location /identity/v3/OS-FEDERATION/identity_providers/testshib/protocols/mapped/auth>
+    ShibRequestSetting requireSession 1
+    AuthType shibboleth
+    ShibExportAssertion Off
+    Require valid-user
+
+    <IfVersion < 2.4>
+        ShibRequireSession On
+        ShibRequireAll On
+   </IfVersion>
+</Location>
diff --git a/devstack/files/federation/shibboleth2.xml b/devstack/files/federation/shibboleth2.xml
new file mode 100644
index 000000000..fc5138cd8
--- /dev/null
+++ b/devstack/files/federation/shibboleth2.xml
@@ -0,0 +1,77 @@
+<!--
+This is an example shibboleth2.xml generated for you by TestShib.  It's reduced and recommended
+specifically for testing.  You don't need to change anything, but you may want to explore the file
+to learn about how your SP works.  Uncomment attributes in your attribute-map.xml file to test them.
+
+If you want to test advanced functionality, start from the distribution shibboleth2.xml and add the
+MetadataProvider, the right entityID, and a properly configured SSO element.  More information:
+
+https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPConfiguration
+-->
+
+<SPConfig xmlns="urn:mace:shibboleth:2.0:native:sp:config" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
+    clockSkew="1800">
+
+    <!-- The entityID is the name TestShib made for your SP. -->
+    <ApplicationDefaults entityID="http://%HOST_IP%/shibboleth">
+
+        <!-- You should use secure cookies if at all possible.  See cookieProps in this Wiki article. -->
+        <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPSessions -->
+        <Sessions lifetime="28800" timeout="3600" checkAddress="false" relayState="ss:mem" handlerSSL="false">
+
+            <!-- Triggers a login request directly to the TestShib IdP. -->
+            <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPServiceSSO -->
+            <SSO entityID="https://idp.testshib.org/idp/shibboleth" ECP="true">
+                SAML2 SAML1
+            </SSO>
+
+            <!-- SAML and local-only logout. -->
+            <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPServiceLogout -->
+            <Logout>SAML2 Local</Logout>
+
+            <!--
+                Handlers allow you to interact with the SP and gather more information.  Try them out!
+                Attribute values received by the SP through SAML will be visible at:
+                http://http@-HOSTNAME-@72@-HOSTNAME-@57@-HOSTNAME-@57128.31.25.69@-HOSTNAME-@725000/Shibboleth.sso/Session
+            -->
+
+            <!-- Extension service that generates "approximate" metadata based on SP configuration. -->
+            <Handler type="MetadataGenerator" Location="/Metadata" signing="false"/>
+
+            <!-- Status reporting service. -->
+            <Handler type="Status" Location="/Status" acl="127.0.0.1 ::1"/>
+
+            <!-- Session diagnostic service. -->
+            <Handler type="Session" Location="/Session" showAttributeValues="true"/>
+
+            <!-- JSON feed of discovery information. -->
+            <Handler type="DiscoveryFeed" Location="/DiscoFeed"/>
+
+        </Sessions>
+
+        <!-- Error pages to display to yourself if something goes horribly wrong. -->
+        <Errors supportContact="root@localhost" logoLocation="/shibboleth-sp/logo.jpg"
+                styleSheet="/shibboleth-sp/main.css"/>
+
+        <!-- Loads and trusts a metadata file that describes only the Testshib IdP and how to communicate with it. -->
+        <MetadataProvider type="XML" uri="http://www.testshib.org/metadata/testshib-providers.xml"
+             backingFilePath="testshib-two-idp-metadata.xml" reloadInterval="180000" />
+
+        <!-- Attribute and trust options you shouldn't need to change. -->
+        <AttributeExtractor type="XML" validate="true" path="attribute-map.xml"/>
+        <AttributeResolver type="Query" subjectMatch="true"/>
+        <AttributeFilter type="XML" validate="true" path="attribute-policy.xml"/>
+
+        <!-- Your SP generated these credentials.  They're used to talk to IdP's. -->
+        <CredentialResolver type="File" key="sp-key.pem" certificate="sp-cert.pem"/>
+
+    </ApplicationDefaults>
+
+    <!-- Security policies you shouldn't change unless you know what you're doing. -->
+    <SecurityPolicyProvider type="XML" validate="true" path="security-policy.xml"/>
+
+    <!-- Low-level configuration about protocols and bindings available for use. -->
+    <ProtocolProvider type="XML" validate="true" reloadChanges="false" path="protocols.xml"/>
+
+</SPConfig>
+
diff --git a/devstack/lib/federation.sh b/devstack/lib/federation.sh
new file mode 100644
index 000000000..4f33bfe84
--- /dev/null
+++ b/devstack/lib/federation.sh
@@ -0,0 +1,74 @@
+# Copyright 2016 Massachusetts Open Cloud
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+function install_federation {
+    if is_ubuntu; then
+        install_package libapache2-mod-shib2
+
+        # Create a new keypair for Shibboleth
+        sudo shib-keygen -f
+
+        # Enable the Shibboleth module for Apache
+        sudo a2enmod shib2
+    else
+        # Note(knikolla): For CentOS/RHEL, installing shibboleth is tricky
+        # It requires adding a separate repo not officially supported
+        echo "Skipping installation of shibboleth for non ubuntu host"
+    fi
+}
+
+function configure_federation {
+    local keystone_apache_conf=$(apache_site_config_for keystone)
+
+    # Add WSGIScriptAlias directive to vhost configuration for port 5000
+    sudo sed -i -e "
+        /<VirtualHost \*:5000>/r $KEYSTONE_PLUGIN/files/federation/shib_apache_alias.txt
+    " $keystone_apache_conf
+
+    # Append to the keystone.conf vhost file a <Location> directive for the Shibboleth module
+    # and a <Location> directive for the identity provider
+    cat $KEYSTONE_PLUGIN/files/federation/shib_apache_handler.txt | sudo tee -a $keystone_apache_conf
+    sudo sed -i -e "s|%IDP_ID%|$IDP_ID|g;" $keystone_apache_conf
+
+    # Copy a templated /etc/shibboleth/shibboleth2.xml file...
+    sudo cp $KEYSTONE_PLUGIN/files/federation/shibboleth2.xml /etc/shibboleth/shibboleth2.xml
+    # ... and replace the %HOST_IP% placeholder with the host ip
+    sudo sed -i -e "s|%HOST_IP%|$HOST_IP|g;" /etc/shibboleth/shibboleth2.xml
+
+    restart_service shibd
+
+    # Enable the mapped auth method in /etc/keystone.conf
+    iniset $KEYSTONE_CONF auth methods "external,password,token,mapped"
+    # Specify the header that contains information about the identity provider
+    iniset $KEYSTONE_CONF mapped remote_id_attribute "Shib-Identity-Provider"
+}
+
+function register_federation {
+    local federated_domain=$(get_or_create_domain federated_domain)
+    local federated_project=$(get_or_create_project federated_project federated_domain)
+    local federated_users=$(get_or_create_group federated_users federated_domain)
+    local member_role=$(get_or_create_role Member)
+
+    openstack role add --group $federated_users --domain $federated_domain $member_role
+    openstack role add --group $federated_users --project $federated_project $member_role
+}
+
+function uninstall_federation {
+    if is_ubuntu; then
+        uninstall_package libapache2-mod-shib2
+        sudo rm -rf /etc/shibboleth
+    else
+        echo "Skipping uninstallation of shibboleth for non ubuntu host"
+    fi
+}
diff --git a/devstack/plugin.sh b/devstack/plugin.sh
index b194bc18f..a1158881d 100644
--- a/devstack/plugin.sh
+++ b/devstack/plugin.sh
@@ -13,6 +13,9 @@
 # License for the specific language governing permissions and limitations
 # under the License.
 
+KEYSTONE_PLUGIN=$DEST/keystone/devstack
+source $KEYSTONE_PLUGIN/lib/federation.sh
+
 # For more information on Devstack plugins, including a more detailed
 # explanation on when the different steps are executed please see:
 # http://docs.openstack.org/developer/devstack/plugins.html
@@ -20,15 +23,24 @@
 if [[ "$1" == "stack" && "$2" == "install" ]]; then
     # This phase is executed after the projects have been installed
     echo "Keystone plugin - Install phase"
+    if is_service_enabled keystone-saml2-federation; then
+        install_federation
+    fi
 
 elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
     # This phase is executed after the projects have been configured and
     # before they are started
     echo "Keystone plugin - Post-config phase"
+    if is_service_enabled keystone-saml2-federation; then
+        configure_federation
+    fi
 
 elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
     # This phase is executed after the projects have been started
     echo "Keystone plugin - Extra phase"
+    if is_service_enabled keystone-saml2-federation; then
+        register_federation
+    fi
 fi
 
 if [[ "$1" == "unstack" ]]; then
@@ -40,5 +52,7 @@ fi
 if [[ "$1" == "clean" ]]; then
     # Called by clean.sh after the "unstack" phase
     # Undo what was performed during the "install" phase
-    :
+    if is_service_enabled keystone-saml2-federation; then
+        uninstall_federation
+    fi
 fi