Read Octet Stream

Join GitHub today

GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.

Sign up New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comments

commented May 2, 2017

My understanding is that the Java SDK sets by default a Content-Type: application/octet-stream header if none is provided by the user when adding an object.
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L4147

On the other hand, I've noticed that files uploaded via AWS CLI's s3api put-object .. get assigned a Content-Type of binary/octet-stream.
I suppose that this value is assigned by the S3 backend, because I couldn't find any Content-Type header set in the PUT request according to --debug logs.

Uploading a file via the S3 Web console produces a binary/octet-stream MIME type as well.
(For the record, Google Cloud Storage also uses binary/octet-stream.)

I was wondering if you had any comment on this difference between application/octet-stream and binary/octet-stream?

referenced this issue May 2, 2017

Merged

[MODE-2692] Return null MIME type in case of 'application/octet-stream' #1656

commented May 2, 2017

Hi don't believe the CLI actually sets binary/octet-stream. You can verify this by issuing a command like aws s3 cp MY_BINARY_FILE s3://MY_BUCKET --debug and see that it does not send a Content-Type header.

Per the S3 docs, the default is binary/octet-stream if the header is not set: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html

commented May 2, 2017

Oops sorry looks like I didn't read your question close enough! I'm not sure what the exact reason is for the discrepancy but I suspect it's because application/octet-stream is registered with the IANA, and binary/octet-stream does not seem to be. In any case, I don't think it's possible for us to change this now because it would be subtle breaking change for customers.

commented May 3, 2017

Hum, if the default behavior can't be changed (what kind of 'subtle change' do you expect by the way?), what about an option to specify the default MIME type?
Or is the only solution for the API user to explicitly set binary/octet-stream for each PUT operation?

commented May 3, 2017

As far as breaking change, customers may be relying on the fact that the default MIME for objects uploaded using the Java SDK to be application/octet-stream rather than binary/octet-stream.

Do you expect all of your objects to have binary/octet-stream? If so, you can use ClientConfiguration#withHeader. Note that this will override the value set on ObjectMetadata#setContentType so it won't work if the you need to be able to override the default value.

Another option might be a custom RequestHandler

We can also look into adding a default object MIME if that would be easiest for you.

commented May 3, 2017

Thanks for the tips on the various methods available.
I have full control of the code that uses the API so I can explicitly use ObjectMetadata#setContentType for each call, that's not a big deal.
Yet, I am curious what you meant in your last sentence.

commented May 3, 2017
edited

Are you referring to

We can also look into adding a default object MIME if that would be easiest for you. Siemens step 7 software price.

?

I was imagining just a new option like setDefaultObjectMimeType on S3ClientOptions that works sort of like ClientConfiguration#withHeader but at the S3 level so you can change mimetype per PUT call.

commented May 3, 2017

Yes, I think such an option would be useful. That would even give some API visibility to the fact that the Java SDK uses a 'non-standard' default MIME type.

commented May 4, 2017

By the way, speaking of 'application/octet-stream' header, I saw in debug traces that (all?) HTTP requests sent to S3 contain such a header, even in case of a GET or HEAD.
I suppose that's not forbidden per se according to the spec, but I find it strange.

assigned dagnirMay 15, 2017

referenced this issue Jan 13, 2019

Open

Crash when guess_extension() returns None #37

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment