Suspicious Operation Django
Question:
I’ve been running into a problem while trying to delete uploaded images.
The error is along these lines:
SuspiciousOperation: Attempted access to '/media/artists/12-stones/154339.jpg' denied.
After reading around it looks like the error is due to the fact that it’s looking for the image in the wrong place (notice first slash, /media/ doesn’t exist on the filesystem)
My MEDIA_ROOT and MEDIA_URL are:
MEDIA_ROOT = '/home/tsoporan/site/media/'
MEDIA_URL = "/media/
My models upload_to parameter is passed this function:
def get_artist_path(instance, filename):
return os.path.join('artists', slugify(instance.name), filename)
My questions are:
1) How can I fix this problem for future uploads?
2) Is it possible to fix my current images’ paths without having to reupload?
Regards,
Titus
Answers:
Well, a little grepping around in the code shows that there may be a deeper error message that got homogenized along the way.
in django/core/files/storage.py, line 210 (this is in 1.1.1) we have:
def path(self, name):
try:
path = safe_join(self.location, name)
except ValueError:
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
return smart_str(os.path.normpath(path))
So the error has to be coming out of safe_join().
In django/utils/_os.py, we have the following. Note the ValueError it throws on the third to last line:
===========================
def safe_join(base, *paths):
"""
Joins one or more path components to the base path component intelligently.
Returns a normalized, absolute version of the final path.
The final path must be located inside of the base path component (otherwise
a ValueError is raised).
"""
# We need to use normcase to ensure we don't false-negative on case
# insensitive operating systems (like Windows).
base = force_unicode(base)
paths = [force_unicode(p) for p in paths]
final_path = normcase(abspathu(join(base, *paths)))
base_path = normcase(abspathu(base))
base_path_len = len(base_path)
# Ensure final_path starts with base_path and that the next character after
# the final path is os.sep (or nothing, in which case final_path must be
# equal to base_path).
if not final_path.startswith(base_path)
or final_path[base_path_len:base_path_len+1] not in ('', sep):
raise ValueError('the joined path is located outside of the base path'
' component')
return final_path
==================
Hmmm, “The joined path is located outside of the base path component”. Now there are a couple of calls to abspathu() in there (which is defined just above this routine and is different for NT than for other OSes). abspathu() converts all non-absolute paths to absolute by tacking on os.cwdu(), the current working directory.
Question: By any chance do you have a symlink (symbolic link) to your media directory? In other words, it’s not a direct child of the project directory? I don’t know if this is a valid question, it just popped out of my head.
Question: What are the values of self.location and name that are being passed to safe_join()?
Wild-ass-guess: is self.location empty?
Another wild-ass-guess: did MEDIA_ROOT somehow get changed to /media/?
If you have your own copy of Django installed (it’s not hard to do), trying putting some print statements in these routines and then run it as the development server. The print output will go to the console.
Update: Hmmm. You said “2) The values for self.location and name are: /home/tsoporan/site/media and /media/albums/anthem-for-the-underdog/30103635.jpg”
Does the following path make any sense?
"/home/tsoporan/site/media/media/albums/anthem-for-the-underdog"
Note the …/media/media/… in there.
Also, what OS is this? Django rev?
Ah figured it out, slightly embarrassing, but it turns out the error was higher up. I was plugging these images in by a script and while going over it again realized that my paths started with /media/.
Now I have about 4000 images with wrong paths … is there a way to somehow amend the paths for all these images? Or will the need to be reuploaded?
Thanks everyone, apologies for my mistake.
You really should just ask a new question on this. Try the following:
Write a standalone django script that looks something like this:
from django.core.management import setup_environ
from mysite import settings
setup_environ(settings)
from django.db import transaction
from app.models import Album # or whatever your model name is
for a in Album.objects.all():
# Do something to cleanup the filename.
# NOTE! This will not move the files, just change the value in the field.
a.filename = re.sub(r'^/media', '', a.filename)
a.save()
transaction.commit_unless_managed() # flush all changes
This bug is fixed in django 1.2.5. See http://docs.djangoproject.com/en/dev/releases/1.2.5/#filefield-no-longer-deletes-files
I got this error when I put a leading slash in the upload_to definition.
BAD
pic = models.ImageField(upload_to="/uploads/product_images/")
GOOD
pic = models.ImageField(upload_to="uploads/product_images/")
if you want use other location ，such as /data/images/myfile/ , you should set you MEDIA_ROOT to /data/images in django settings.py file .
As a note for others this issue can be caused when you have a double ‘//’ in the static file resource you are looking for.
{{ STATIC_URL }}/style.css # Causes the issue it should be
{{ STATIC_URL }}style.css
I got this error too. Debugging through I found that the following exception is being raised.
SuspiciousOperation(u"Attempted access to '2015-03-19-08:29:51-2-f8945842891244629dfd0c0af4c72a9c.pdf' denied.",)
BTW, I am using django-storages (v1.1.8) to store my media files onto S3 (using S3boto backend). I am using django 1.7.6.
But if I switch to storing with file name with out colons (:) it seems to work. I haven’t figured out yet what is the root cause. Just posting this in case this is helpful to somebody else. Apparently, django or django-storages does not like filenames with colons.
I found out, by using dumb print statements, that some media files have /media prefixed in their url paths. While the default storage option handles that, there is a problem if you use S3BotoStorage from django-storages.
So I fixed it by overriding the _normalize_name (guided by the answer by @peter-rowell):
class MediaStorage(FixedUrlBotoStorage):
location = settings.MEDIAFILES_LOCATION
# Overriding function because some media files are stored with '/media' prefixed (which causes problems)
def _normalize_name(self, name):
if name.startswith('/media'):
name = name.lstrip('/media')
return super(MediaStorage, self)._normalize_name(name)
Use SimpleUploadedFile if your temporary file is not part of the MEDIA_ROOT folder. This won’t throw a SuspiciousOperation error:
upload_file = SimpleUploadedFile(name=basename(out_file), content=open(out_file, 'rb').read())
object = YourModel.objects.create(file=upload_file)
Use File if your temporary file is already a part of MEDIA_ROOT. This is useful if you want to link an existing Django file to an object.
object = YourModel.objects.create(file=File(open(file_path, 'rb')))
I fixed this in a very simple way, go to utils.py in this folder
djklibsite-packagesdjangocorefilesutils.py
(djk is the name of the virtualenv)
In the file just make line 7 and line 8 python comments and that’s it, job’s done.
I’ve been running into a problem while trying to delete uploaded images.
The error is along these lines:
SuspiciousOperation: Attempted access to '/media/artists/12-stones/154339.jpg' denied.
After reading around it looks like the error is due to the fact that it’s looking for the image in the wrong place (notice first slash, /media/ doesn’t exist on the filesystem)
My MEDIA_ROOT and MEDIA_URL are:
MEDIA_ROOT = '/home/tsoporan/site/media/'
MEDIA_URL = "/media/
My models upload_to parameter is passed this function:
def get_artist_path(instance, filename):
return os.path.join('artists', slugify(instance.name), filename)
My questions are:
1) How can I fix this problem for future uploads?
2) Is it possible to fix my current images’ paths without having to reupload?
Regards,
Titus
Well, a little grepping around in the code shows that there may be a deeper error message that got homogenized along the way.
in django/core/files/storage.py, line 210 (this is in 1.1.1) we have:
def path(self, name):
try:
path = safe_join(self.location, name)
except ValueError:
raise SuspiciousOperation("Attempted access to '%s' denied." % name)
return smart_str(os.path.normpath(path))
So the error has to be coming out of safe_join().
In django/utils/_os.py, we have the following. Note the ValueError it throws on the third to last line:
===========================
def safe_join(base, *paths):
"""
Joins one or more path components to the base path component intelligently.
Returns a normalized, absolute version of the final path.
The final path must be located inside of the base path component (otherwise
a ValueError is raised).
"""
# We need to use normcase to ensure we don't false-negative on case
# insensitive operating systems (like Windows).
base = force_unicode(base)
paths = [force_unicode(p) for p in paths]
final_path = normcase(abspathu(join(base, *paths)))
base_path = normcase(abspathu(base))
base_path_len = len(base_path)
# Ensure final_path starts with base_path and that the next character after
# the final path is os.sep (or nothing, in which case final_path must be
# equal to base_path).
if not final_path.startswith(base_path)
or final_path[base_path_len:base_path_len+1] not in ('', sep):
raise ValueError('the joined path is located outside of the base path'
' component')
return final_path
==================
Hmmm, “The joined path is located outside of the base path component”. Now there are a couple of calls to abspathu() in there (which is defined just above this routine and is different for NT than for other OSes). abspathu() converts all non-absolute paths to absolute by tacking on os.cwdu(), the current working directory.
Question: By any chance do you have a symlink (symbolic link) to your media directory? In other words, it’s not a direct child of the project directory? I don’t know if this is a valid question, it just popped out of my head.
Question: What are the values of self.location and name that are being passed to safe_join()?
Wild-ass-guess: is self.location empty?
Another wild-ass-guess: did MEDIA_ROOT somehow get changed to /media/?
If you have your own copy of Django installed (it’s not hard to do), trying putting some print statements in these routines and then run it as the development server. The print output will go to the console.
Update: Hmmm. You said “2) The values for self.location and name are: /home/tsoporan/site/media and /media/albums/anthem-for-the-underdog/30103635.jpg”
Does the following path make any sense?
"/home/tsoporan/site/media/media/albums/anthem-for-the-underdog"
Note the …/media/media/… in there.
Also, what OS is this? Django rev?
Ah figured it out, slightly embarrassing, but it turns out the error was higher up. I was plugging these images in by a script and while going over it again realized that my paths started with /media/.
Now I have about 4000 images with wrong paths … is there a way to somehow amend the paths for all these images? Or will the need to be reuploaded?
Thanks everyone, apologies for my mistake.
You really should just ask a new question on this. Try the following:
Write a standalone django script that looks something like this:
from django.core.management import setup_environ
from mysite import settings
setup_environ(settings)
from django.db import transaction
from app.models import Album # or whatever your model name is
for a in Album.objects.all():
# Do something to cleanup the filename.
# NOTE! This will not move the files, just change the value in the field.
a.filename = re.sub(r'^/media', '', a.filename)
a.save()
transaction.commit_unless_managed() # flush all changes
This bug is fixed in django 1.2.5. See http://docs.djangoproject.com/en/dev/releases/1.2.5/#filefield-no-longer-deletes-files
I got this error when I put a leading slash in the upload_to definition.
BAD
pic = models.ImageField(upload_to="/uploads/product_images/")
GOOD
pic = models.ImageField(upload_to="uploads/product_images/")
if you want use other location ，such as /data/images/myfile/ , you should set you MEDIA_ROOT to /data/images in django settings.py file .
As a note for others this issue can be caused when you have a double ‘//’ in the static file resource you are looking for.
{{ STATIC_URL }}/style.css # Causes the issue it should be
{{ STATIC_URL }}style.css
I got this error too. Debugging through I found that the following exception is being raised.
SuspiciousOperation(u"Attempted access to '2015-03-19-08:29:51-2-f8945842891244629dfd0c0af4c72a9c.pdf' denied.",)
BTW, I am using django-storages (v1.1.8) to store my media files onto S3 (using S3boto backend). I am using django 1.7.6.
But if I switch to storing with file name with out colons (:) it seems to work. I haven’t figured out yet what is the root cause. Just posting this in case this is helpful to somebody else. Apparently, django or django-storages does not like filenames with colons.
I found out, by using dumb print statements, that some media files have /media prefixed in their url paths. While the default storage option handles that, there is a problem if you use S3BotoStorage from django-storages.
So I fixed it by overriding the _normalize_name (guided by the answer by @peter-rowell):
class MediaStorage(FixedUrlBotoStorage):
location = settings.MEDIAFILES_LOCATION
# Overriding function because some media files are stored with '/media' prefixed (which causes problems)
def _normalize_name(self, name):
if name.startswith('/media'):
name = name.lstrip('/media')
return super(MediaStorage, self)._normalize_name(name)
Use SimpleUploadedFile if your temporary file is not part of the MEDIA_ROOT folder. This won’t throw a SuspiciousOperation error:
upload_file = SimpleUploadedFile(name=basename(out_file), content=open(out_file, 'rb').read())
object = YourModel.objects.create(file=upload_file)
Use File if your temporary file is already a part of MEDIA_ROOT. This is useful if you want to link an existing Django file to an object.
object = YourModel.objects.create(file=File(open(file_path, 'rb')))
I fixed this in a very simple way, go to utils.py in this folder
djklibsite-packagesdjangocorefilesutils.py
(djk is the name of the virtualenv)
In the file just make line 7 and line 8 python comments and that’s it, job’s done.