Google Cloud Storage file stuck in time after multiple updates/deletions
Question:
I have a cloud storage file frozen in time and will not budge. So far I have:
-
recollected static files (Django) and did an rsync to update
-
deleted the static file in question, recollected static files and did an rsync to update
-
deleted the file in the cloud storage console and re-rsynced
-
deleted the entire bucket and made another bucket with the same name, and rsynced all 85 static files onto it again
The first few times I changed the static file, collected them, and rsynced, it worked instantly. Now I have this one line
var URL = "http://%/api/recipes/delete/&/".replace("%", API_BASE).replace("&", recipe_id);
stuck in time, frozen in this bucket. In Django, my file correctly reads
var URL = "/api/recipes/delete/&/".replace("&", recipe_id);
The change is checked into git.
This storage API has not updated at all, and the file was still being served even after I deleted the file. The file was still being served after I deleted the entire bucket, which seems like a bug. The file is:
http://storage.googleapis.com/homebrew-api-static-bucket/static/main/index.js
Yet if I view the file from the Cloud Storage console, I get the correct file.
So even though the file is correct in the bucket, the file my app uses is still outdated after over 2 hours
I serve the file in index.html like:
<script src="{% static "main/index.js" %}" type='text/javascript'></script>
In settings.py:
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
STATIC_URL = 'http://storage.googleapis.com/homebrew-bucket/static/'
I’m out of ideas if even deleting and remaking the entire bucket doesn’t change anything.
Answers:
GCS provides strong read-after-write consistency. If you upload a new version of an object and then download it back, you will always get the new version of the object.
However, by default, when a publicly readable object is fetched, GCS responds with a “Cache-Control” header allowing caching for up to 1 hour. Google itself, your web browser, or some firewall between you and Google may decide based on that header to cache your object. You can check to see if this is the problem by fetching the object again and tacking some bogus URL parameter onto the end of the request, like “?avoidTheCaches=1”.
You can prevent this from happening by setting the object’s “cacheControl” property to something like “private” or “max-age=0”. You can set this property as part of an upload. You can also modify the property for an existing object, but if the resource has already been cached, you’ll have to wait out the hour before you’ll see the new version.
You can go to the object in your console, click on "Edit Metadata", then enter what you want in the "Cache control" field. If you want no caching at all, you can add the string no-store.
Documentation on the allowed values is here.
I have a cloud storage file frozen in time and will not budge. So far I have:
-
recollected static files (Django) and did an rsync to update
-
deleted the static file in question, recollected static files and did an rsync to update
-
deleted the file in the cloud storage console and re-rsynced
-
deleted the entire bucket and made another bucket with the same name, and rsynced all 85 static files onto it again
The first few times I changed the static file, collected them, and rsynced, it worked instantly. Now I have this one line
var URL = "http://%/api/recipes/delete/&/".replace("%", API_BASE).replace("&", recipe_id);
stuck in time, frozen in this bucket. In Django, my file correctly reads
var URL = "/api/recipes/delete/&/".replace("&", recipe_id);
The change is checked into git.
This storage API has not updated at all, and the file was still being served even after I deleted the file. The file was still being served after I deleted the entire bucket, which seems like a bug. The file is:
http://storage.googleapis.com/homebrew-api-static-bucket/static/main/index.js
Yet if I view the file from the Cloud Storage console, I get the correct file.
So even though the file is correct in the bucket, the file my app uses is still outdated after over 2 hours
I serve the file in index.html like:
<script src="{% static "main/index.js" %}" type='text/javascript'></script>
In settings.py:
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
STATIC_URL = 'http://storage.googleapis.com/homebrew-bucket/static/'
I’m out of ideas if even deleting and remaking the entire bucket doesn’t change anything.
GCS provides strong read-after-write consistency. If you upload a new version of an object and then download it back, you will always get the new version of the object.
However, by default, when a publicly readable object is fetched, GCS responds with a “Cache-Control” header allowing caching for up to 1 hour. Google itself, your web browser, or some firewall between you and Google may decide based on that header to cache your object. You can check to see if this is the problem by fetching the object again and tacking some bogus URL parameter onto the end of the request, like “?avoidTheCaches=1”.
You can prevent this from happening by setting the object’s “cacheControl” property to something like “private” or “max-age=0”. You can set this property as part of an upload. You can also modify the property for an existing object, but if the resource has already been cached, you’ll have to wait out the hour before you’ll see the new version.
You can go to the object in your console, click on "Edit Metadata", then enter what you want in the "Cache control" field. If you want no caching at all, you can add the string no-store.
Documentation on the allowed values is here.