Custom Authentication for Google Cloud Endpoints (instead of OAuth2)
Question:
We are super excited about App Engine’s support for Google Cloud Endpoints.
That said we don’t use OAuth2 yet and usually authenticate users with username/password
so we can support customers that don’t have Google accounts.
We want to migrate our API over to Google Cloud Endpoints because of all the benefits we then get for free (API Console, Client Libraries, robustness, …) but our main question is …
How to add custom authentication to cloud endpoints where we previously check for a valid user session + CSRF token in our existing API.
Is there an elegant way to do this without adding stuff like session information and CSRF tokens to the protoRPC messages?
Answers:
From my understanding Google Cloud Endpoints provides a way to implement a (RESTful?) API and to generate a mobile client library. Authentication in this case would be OAuth2. OAuth2 provides different ‘flows’, some of which support mobile clients.
In the case of authentication using a principal and credentials (username and password) this doesn’t seem like a good fit. I honestly think you would be better off by using OAuth2.
Implementing a custom OAuth2 flow to support your case is an approach that could work but is very error prone.
I haven’t worked with OAuth2 yet but maybe an ‘API key’ can be created for a user so they can both use the front-end and the back-end through the use of mobile clients.
I’m using webapp2 Authentication system for my entire application. So I tried to reuse this for Google Cloud Authentication and I get it!
webapp2_extras.auth uses webapp2_extras.sessions to store auth information. And it this session could be stored in 3 different formats: securecookie, datastore or memcache.
Securecookie is the default format and which I’m using. I consider it secure enough as webapp2 auth system is used for a lot of GAE application running in production enviroment.
So I decode this securecookie and reuse it from GAE Endpoints. I don’t know if this could generate some secure problem (I hope not) but maybe @bossylobster could say if it is ok looking at security side.
My Api:
import Cookie
import logging
import endpoints
import os
from google.appengine.ext import ndb
from protorpc import remote
import time
from webapp2_extras.sessions import SessionDict
from web.frankcrm_api_messages import IdContactMsg, FullContactMsg, ContactList, SimpleResponseMsg
from web.models import Contact, User
from webapp2_extras import sessions, securecookie, auth
import config
__author__ = 'Douglas S. Correa'
TOKEN_CONFIG = {
'token_max_age': 86400 * 7 * 3,
'token_new_age': 86400,
'token_cache_age': 3600,
}
SESSION_ATTRIBUTES = ['user_id', 'remember',
'token', 'token_ts', 'cache_ts']
SESSION_SECRET_KEY = '9C3155EFEEB9D9A66A22EDC16AEDA'
@endpoints.api(name='frank', version='v1',
description='FrankCRM API')
class FrankApi(remote.Service):
user = None
token = None
@classmethod
def get_user_from_cookie(cls):
serializer = securecookie.SecureCookieSerializer(SESSION_SECRET_KEY)
cookie_string = os.environ.get('HTTP_COOKIE')
cookie = Cookie.SimpleCookie()
cookie.load(cookie_string)
session = cookie['session'].value
session_name = cookie['session_name'].value
session_name_data = serializer.deserialize('session_name', session_name)
session_dict = SessionDict(cls, data=session_name_data, new=False)
if session_dict:
session_final = dict(zip(SESSION_ATTRIBUTES, session_dict.get('_user')))
_user, _token = cls.validate_token(session_final.get('user_id'), session_final.get('token'),
token_ts=session_final.get('token_ts'))
cls.user = _user
cls.token = _token
@classmethod
def user_to_dict(cls, user):
"""Returns a dictionary based on a user object.
Extra attributes to be retrieved must be set in this module's
configuration.
:param user:
User object: an instance the custom user model.
:returns:
A dictionary with user data.
"""
if not user:
return None
user_dict = dict((a, getattr(user, a)) for a in [])
user_dict['user_id'] = user.get_id()
return user_dict
@classmethod
def get_user_by_auth_token(cls, user_id, token):
"""Returns a user dict based on user_id and auth token.
:param user_id:
User id.
:param token:
Authentication token.
:returns:
A tuple ``(user_dict, token_timestamp)``. Both values can be None.
The token timestamp will be None if the user is invalid or it
is valid but the token requires renewal.
"""
user, ts = User.get_by_auth_token(user_id, token)
return cls.user_to_dict(user), ts
@classmethod
def validate_token(cls, user_id, token, token_ts=None):
"""Validates a token.
Tokens are random strings used to authenticate temporarily. They are
used to validate sessions or service requests.
:param user_id:
User id.
:param token:
Token to be checked.
:param token_ts:
Optional token timestamp used to pre-validate the token age.
:returns:
A tuple ``(user_dict, token)``.
"""
now = int(time.time())
delete = token_ts and ((now - token_ts) > TOKEN_CONFIG['token_max_age'])
create = False
if not delete:
# Try to fetch the user.
user, ts = cls.get_user_by_auth_token(user_id, token)
if user:
# Now validate the real timestamp.
delete = (now - ts) > TOKEN_CONFIG['token_max_age']
create = (now - ts) > TOKEN_CONFIG['token_new_age']
if delete or create or not user:
if delete or create:
# Delete token from db.
User.delete_auth_token(user_id, token)
if delete:
user = None
token = None
return user, token
@endpoints.method(IdContactMsg, ContactList,
path='contact/list', http_method='GET',
name='contact.list')
def list_contacts(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
model_list = Contact.query().fetch(20)
contact_list = []
for contact in model_list:
contact_list.append(contact.to_full_contact_message())
return ContactList(contact_list=contact_list)
@endpoints.method(FullContactMsg, IdContactMsg,
path='contact/add', http_method='POST',
name='contact.add')
def add_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
new_contact = Contact.put_from_message(request)
logging.info(new_contact.key.id())
return IdContactMsg(id=new_contact.key.id())
@endpoints.method(FullContactMsg, IdContactMsg,
path='contact/update', http_method='POST',
name='contact.update')
def update_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
new_contact = Contact.put_from_message(request)
logging.info(new_contact.key.id())
return IdContactMsg(id=new_contact.key.id())
@endpoints.method(IdContactMsg, SimpleResponseMsg,
path='contact/delete', http_method='POST',
name='contact.delete')
def delete_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
if request.id:
contact_to_delete_key = ndb.Key(Contact, request.id)
if contact_to_delete_key.get():
contact_to_delete_key.delete()
return SimpleResponseMsg(success=True)
return SimpleResponseMsg(success=False)
APPLICATION = endpoints.api_server([FrankApi],
restricted=False)
I wrote a custom python authentication library called Authtopus that may be of interest to anyone looking for a solution to this problem: https://github.com/rggibson/Authtopus
Authtopus supports basic username and password registrations and logins, as well as social logins via Facebook or Google (more social providers could probably be added without too much hassle too). User accounts are merged according to verified email addresses, so if a user first registers by username and password, then later uses a social login, and the verified email addresses of the accounts match up, then no separate User account is created.
I did not coded it yet, but it imagined next way:
-
When server receives login request it look up username/password in datastore. In case user not found server responds with some error object that contains appropriate message like “User doesn’t exist” or like. In case found it stored in FIFO kind of collection (cache) with limited size like 100 (or 1000 or 10000).
-
On successful login request server returns to client sessionid like “;LKJLK345345LKJLKJSDF53KL”. Can be Base64 encoded username:password.
Client stores it in Cookie named “authString” or “sessionid” (or something less eloquent) with 30 min (any) expiration.
-
With each request after login client sends Autorization header that it takes from cookie. Each time cookie taken, it renewed — so it never expires while user active.
-
On server side we will have AuthFilter that will check presence of Authorization header in each request (exclude login, signup, reset_password). If no such header found, filter returns response to client with status code 401 (client shows login screen to user). If header found filter first checks presence of user in the cache, after in datastore and if user found — does nothing (request handled by appropriate method), not found — 401.
Above architecture allows to keep server stateless but still have auto disconnecting sessions.
We are super excited about App Engine’s support for Google Cloud Endpoints.
That said we don’t use OAuth2 yet and usually authenticate users with username/password
so we can support customers that don’t have Google accounts.
We want to migrate our API over to Google Cloud Endpoints because of all the benefits we then get for free (API Console, Client Libraries, robustness, …) but our main question is …
How to add custom authentication to cloud endpoints where we previously check for a valid user session + CSRF token in our existing API.
Is there an elegant way to do this without adding stuff like session information and CSRF tokens to the protoRPC messages?
From my understanding Google Cloud Endpoints provides a way to implement a (RESTful?) API and to generate a mobile client library. Authentication in this case would be OAuth2. OAuth2 provides different ‘flows’, some of which support mobile clients.
In the case of authentication using a principal and credentials (username and password) this doesn’t seem like a good fit. I honestly think you would be better off by using OAuth2.
Implementing a custom OAuth2 flow to support your case is an approach that could work but is very error prone.
I haven’t worked with OAuth2 yet but maybe an ‘API key’ can be created for a user so they can both use the front-end and the back-end through the use of mobile clients.
I’m using webapp2 Authentication system for my entire application. So I tried to reuse this for Google Cloud Authentication and I get it!
webapp2_extras.auth uses webapp2_extras.sessions to store auth information. And it this session could be stored in 3 different formats: securecookie, datastore or memcache.
Securecookie is the default format and which I’m using. I consider it secure enough as webapp2 auth system is used for a lot of GAE application running in production enviroment.
So I decode this securecookie and reuse it from GAE Endpoints. I don’t know if this could generate some secure problem (I hope not) but maybe @bossylobster could say if it is ok looking at security side.
My Api:
import Cookie
import logging
import endpoints
import os
from google.appengine.ext import ndb
from protorpc import remote
import time
from webapp2_extras.sessions import SessionDict
from web.frankcrm_api_messages import IdContactMsg, FullContactMsg, ContactList, SimpleResponseMsg
from web.models import Contact, User
from webapp2_extras import sessions, securecookie, auth
import config
__author__ = 'Douglas S. Correa'
TOKEN_CONFIG = {
'token_max_age': 86400 * 7 * 3,
'token_new_age': 86400,
'token_cache_age': 3600,
}
SESSION_ATTRIBUTES = ['user_id', 'remember',
'token', 'token_ts', 'cache_ts']
SESSION_SECRET_KEY = '9C3155EFEEB9D9A66A22EDC16AEDA'
@endpoints.api(name='frank', version='v1',
description='FrankCRM API')
class FrankApi(remote.Service):
user = None
token = None
@classmethod
def get_user_from_cookie(cls):
serializer = securecookie.SecureCookieSerializer(SESSION_SECRET_KEY)
cookie_string = os.environ.get('HTTP_COOKIE')
cookie = Cookie.SimpleCookie()
cookie.load(cookie_string)
session = cookie['session'].value
session_name = cookie['session_name'].value
session_name_data = serializer.deserialize('session_name', session_name)
session_dict = SessionDict(cls, data=session_name_data, new=False)
if session_dict:
session_final = dict(zip(SESSION_ATTRIBUTES, session_dict.get('_user')))
_user, _token = cls.validate_token(session_final.get('user_id'), session_final.get('token'),
token_ts=session_final.get('token_ts'))
cls.user = _user
cls.token = _token
@classmethod
def user_to_dict(cls, user):
"""Returns a dictionary based on a user object.
Extra attributes to be retrieved must be set in this module's
configuration.
:param user:
User object: an instance the custom user model.
:returns:
A dictionary with user data.
"""
if not user:
return None
user_dict = dict((a, getattr(user, a)) for a in [])
user_dict['user_id'] = user.get_id()
return user_dict
@classmethod
def get_user_by_auth_token(cls, user_id, token):
"""Returns a user dict based on user_id and auth token.
:param user_id:
User id.
:param token:
Authentication token.
:returns:
A tuple ``(user_dict, token_timestamp)``. Both values can be None.
The token timestamp will be None if the user is invalid or it
is valid but the token requires renewal.
"""
user, ts = User.get_by_auth_token(user_id, token)
return cls.user_to_dict(user), ts
@classmethod
def validate_token(cls, user_id, token, token_ts=None):
"""Validates a token.
Tokens are random strings used to authenticate temporarily. They are
used to validate sessions or service requests.
:param user_id:
User id.
:param token:
Token to be checked.
:param token_ts:
Optional token timestamp used to pre-validate the token age.
:returns:
A tuple ``(user_dict, token)``.
"""
now = int(time.time())
delete = token_ts and ((now - token_ts) > TOKEN_CONFIG['token_max_age'])
create = False
if not delete:
# Try to fetch the user.
user, ts = cls.get_user_by_auth_token(user_id, token)
if user:
# Now validate the real timestamp.
delete = (now - ts) > TOKEN_CONFIG['token_max_age']
create = (now - ts) > TOKEN_CONFIG['token_new_age']
if delete or create or not user:
if delete or create:
# Delete token from db.
User.delete_auth_token(user_id, token)
if delete:
user = None
token = None
return user, token
@endpoints.method(IdContactMsg, ContactList,
path='contact/list', http_method='GET',
name='contact.list')
def list_contacts(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
model_list = Contact.query().fetch(20)
contact_list = []
for contact in model_list:
contact_list.append(contact.to_full_contact_message())
return ContactList(contact_list=contact_list)
@endpoints.method(FullContactMsg, IdContactMsg,
path='contact/add', http_method='POST',
name='contact.add')
def add_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
new_contact = Contact.put_from_message(request)
logging.info(new_contact.key.id())
return IdContactMsg(id=new_contact.key.id())
@endpoints.method(FullContactMsg, IdContactMsg,
path='contact/update', http_method='POST',
name='contact.update')
def update_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
new_contact = Contact.put_from_message(request)
logging.info(new_contact.key.id())
return IdContactMsg(id=new_contact.key.id())
@endpoints.method(IdContactMsg, SimpleResponseMsg,
path='contact/delete', http_method='POST',
name='contact.delete')
def delete_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
if request.id:
contact_to_delete_key = ndb.Key(Contact, request.id)
if contact_to_delete_key.get():
contact_to_delete_key.delete()
return SimpleResponseMsg(success=True)
return SimpleResponseMsg(success=False)
APPLICATION = endpoints.api_server([FrankApi],
restricted=False)
I wrote a custom python authentication library called Authtopus that may be of interest to anyone looking for a solution to this problem: https://github.com/rggibson/Authtopus
Authtopus supports basic username and password registrations and logins, as well as social logins via Facebook or Google (more social providers could probably be added without too much hassle too). User accounts are merged according to verified email addresses, so if a user first registers by username and password, then later uses a social login, and the verified email addresses of the accounts match up, then no separate User account is created.
I did not coded it yet, but it imagined next way:
-
When server receives login request it look up username/password in datastore. In case user not found server responds with some error object that contains appropriate message like “User doesn’t exist” or like. In case found it stored in FIFO kind of collection (cache) with limited size like 100 (or 1000 or 10000).
-
On successful login request server returns to client sessionid like “;LKJLK345345LKJLKJSDF53KL”. Can be Base64 encoded username:password.
Client stores it in Cookie named “authString” or “sessionid” (or something less eloquent) with 30 min (any) expiration. -
With each request after login client sends Autorization header that it takes from cookie. Each time cookie taken, it renewed — so it never expires while user active.
-
On server side we will have AuthFilter that will check presence of Authorization header in each request (exclude login, signup, reset_password). If no such header found, filter returns response to client with status code 401 (client shows login screen to user). If header found filter first checks presence of user in the cache, after in datastore and if user found — does nothing (request handled by appropriate method), not found — 401.
Above architecture allows to keep server stateless but still have auto disconnecting sessions.