Module: X::Uploader::MediaUpload

Extended by:
MediaUpload
Included in:
MediaUpload
Defined in:
x-uploader/lib/x/uploader/media_upload.rb

Overview

Uploads media files to the X API

Its methods can be called on the module, or on an instance of a class that includes it, which gains its public methods alone: what they call belongs to modules of its own, or is called on the module, as upload calls await_processing, so no method the class defines, under any name, can change an upload.

Constant Summary collapse

DEFAULT_PROCESSING_TIMEOUT =

Default number of seconds await_processing waits for processing to finish before it gives up

600
DEFAULT_CONCURRENCY =

Default number of chunks uploaded at once

4
DEFAULT_CHUNK_SIZE =

Default number of bytes in each chunk of an upload in chunks, 4 megabytes, unless the media needs larger ones to fit the segments the API numbers

Validator::DEFAULT_CHUNK
MAX_CONCURRENCY =

Greatest number of chunks uploaded at once, each of which holds a chunk of up to 5 megabytes and a connection

Validator::MAX_CONCURRENCY

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.await_processing(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT) ⇒ UploadedMedia

Wait for media processing to complete

Media that already says its processing has ended, in success or failure, or that holds an upload response which names no processing, as that of an image does, is returned as it is, without a request, since a check would tell no more than the media holds; await_processing! raises for media that says it failed, without a request too.

Before each check it waits as long as X asked, and at least a second: media an upload returned, or the Hash of its response, which says how long to wait before the first check, is not checked until then, and media given as its identifier, or its identifier and media key alone, which say nothing of its processing, or as media in a state of processing X does not document, is checked at once.

The processing timeout is a deadline, the seconds from when it is called, measured on the monotonic clock, so that it counts the time each check takes, with any wait for a rate limit and any retry the client makes, as well as the waits between them. It gives up once the next check X asks for would come after the deadline, rather than sleep past it, or check before X asks. A check under way at the deadline is let finish, and its status returned if processing has finished, so it can return that much after the deadline.

It waits while the processing is pending or in progress alone, so a status whose processing names no state, or a state X does not document, is returned as it is, neither processing nor ready, rather than checked until the deadline, since X gives no time to check it again at; await_processing! raises for it.

Examples:

Wait for processing

Uploader::MediaUpload.await_processing(media, client: client)

Wait for the processing of media known by its identifier

Uploader::MediaUpload.await_processing("1880028106020515840", client: client)

Wait up to half an hour for a long video

Uploader::MediaUpload.await_processing(media, client: client, processing_timeout: 1800)

Parameters:

  • media (UploadedMedia, Hash, #media_key, String, Integer) —

    the uploaded media, media that has a media key, such as X::Media, the media key, or the media identifier

  • client (Client) —

    the X API client

  • processing_timeout (Integer, Float, nil) (defaults to: DEFAULT_PROCESSING_TIMEOUT) —

    the seconds from now to wait for processing to finish, checks and all, before giving up, or nil to wait for as long as processing takes

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the processing status, or the media given, as uploaded media, if its processing has already ended

Raises:

  • (ArgumentError) —

    if the processing timeout is neither a finite number of seconds of at least 0 nor nil

  • (ArgumentError) —

    if the media given is nil, holds no identifier, or is neither media, a media key, nor a media identifier, or its media key names none

  • (MissingMediaData) —

    if a status response holds no media or carries no body at all

  • (MediaProcessingTimeout) —

    if the media is still processing once the next check would pass the deadline



322
323
324
325
326
327
328
329
330
331
332
333
334
335
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 322

def await_processing(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT)
  Validator.validate_processing_timeout!(processing_timeout)
  uploaded = Utils.uploaded_media(media)
  return uploaded if uploaded.ready? || uploaded.failed?

  deadline, media_id, pending = processing_timeout&.then { |seconds| Utils.seconds_from_now(seconds) }, Utils.media_id(uploaded), (uploaded if uploaded.processing?)
  loop do
    Utils.wait_to_check(pending, deadline:, timeout: processing_timeout) if pending
    status = UploadedMedia.new(Utils.media_data(client.get("media/upload", params: {command: STATUS_COMMAND, media_id:}, **JSON_CLASSES), "of the status check"))
    return status unless status.processing?

    pending = status
  end
end

.await_processing!(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT) ⇒ UploadedMedia

Wait for media processing and raise on failure

Examples:

Wait for processing with error handling

Uploader::MediaUpload.await_processing!(media, client: client)

Parameters:

  • media (UploadedMedia, Hash, #media_key, String, Integer) —

    the uploaded media, media that has a media key, such as X::Media, the media key, or the media identifier

  • client (Client) —

    the X API client

  • processing_timeout (Integer, Float, nil) (defaults to: DEFAULT_PROCESSING_TIMEOUT) —

    the seconds from now to wait for processing to finish, checks and all, before giving up, as #await_processing counts them, or nil to wait for as long as processing takes

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the processing status, or the media given, as uploaded media, if its processing has already succeeded

Raises:

  • (ArgumentError) —

    if the processing timeout is neither a finite number of seconds of at least 0 nor nil

  • (ArgumentError) —

    if the media given is nil, holds no identifier, or is neither media, a media key, nor a media identifier, or its media key names none

  • (MissingMediaData) —

    if a status response holds no media or carries no body at all

  • (MediaProcessingFailed) —

    if media processing failed, or ended in no state X documents, with the status X reported, or the media given, without a request, if it already says its processing failed

  • (MediaProcessingTimeout) —

    if the media is still processing once the next check would pass the deadline



356
357
358
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 356

def await_processing!(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT)
  Utils.processed!(MediaUpload.await_processing(media, client:, processing_timeout:))
end

.chunked_upload(media, client:, media_category: nil, media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY, shared: nil, additional_owners: nil) ⇒ UploadedMedia

Perform a chunked upload for large files

It is the way to upload media without waiting for X to process it: #upload waits for the processing of media X processes, such as a video, and this returns once the upload is finalized, so that the caller can go on while X processes a long video, and wait for it with #await_processing or #await_processing! when it needs it. It uploads in chunks whatever the media, an image as well.

The chunks of an upload in chunks are sent by threads of their own, as many as the concurrency, so the on_response of the client runs on those threads for the response of each chunk, and a hook that reads state kept for the thread that called, such as a Rails CurrentAttributes or a logger of its own, reads that of another thread.

Each chunk is a request of its own, which a rate limit can refuse, and a chunk refused raises ChunkedUploadFailed, since the client retries a request refused for a rate limit only max_rate_limit_retries times, which is 0 by default. A large video, uploaded in many chunks, should be uploaded with a client whose max_rate_limit_retries is set, such as X::Client.new(max_rate_limit_retries: 3), so that a rate limit is waited out, up to the max_rate_limit_wait of the client, rather than fail the upload.

Examples:

Upload a large video

Uploader::MediaUpload.chunked_upload("video.mp4", client: client)

Parameters:

  • media (String, Pathname, IO, StringIO) —

    the path to the media to upload, or an IO open on it

  • client (Client) —

    the X API client

  • media_category (String, Symbol, nil) (defaults to: nil) —

    the media category, in any case, inferred from the media when nil

  • media_type (String, nil) (defaults to: nil) —

    the MIME type of the media, sent as it is given, or inferred from the media and category when nil

  • chunk_size (Integer, nil) (defaults to: nil) —

    the size of each chunk in bytes, of at most 5,242,880, the 5 megabytes the API takes in a segment, or nil for DEFAULT_CHUNK_SIZE, 4,194,304 bytes, or as much more as the media needs to fit the segments the API numbers

  • concurrency (Integer) (defaults to: DEFAULT_CONCURRENCY) —

    the number of chunks uploaded at once, of 1 to MAX_CONCURRENCY

  • shared (Boolean, nil) (defaults to: nil) —

    whether the media is shared, so that it can be sent in more than one direct message, or nil to leave it to the API

  • additional_owners (Array<Integer, String>, nil) (defaults to: nil) —

    the identifiers of the users, other than the one who uploads it, who may use the media, or nil for none

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the upload response

Raises:

  • (ArgumentError) —

    if the media is neither a path nor an IO, or is a String that holds a NUL byte or a line break, as the contents of media given in place of its path do

  • (InvalidMedia) —

    if the file does not exist

  • (InvalidMedia) —

    if the media cannot be read, or is empty, which holds nothing to upload

  • (InvalidMedia) —

    if the media is larger than the API takes of its category, whatever the account: 5 megabytes of an image, 15 of a GIF, and one of subtitles, or larger than the 16 gigabytes it takes of any

  • (ArgumentError) —

    if the media category is invalid, the chunk size is not a positive Integer, is larger than a segment the API takes, or would need more segments than the API numbers, the concurrency is not 1 to MAX_CONCURRENCY, shared is neither true, false, nor nil, or additional_owners is neither nil nor an Array of at least one user identifier

  • (InvalidMediaType) —

    if no media type is given and none can be inferred, or the one the media is, read from its bytes or else from the name of its file, is not one the category takes

  • (MissingMediaData) —

    if the response that initializes the upload holds no media to append the chunks to

  • (ChunkedUploadFailed) —

    if the upload is initialized, but a chunk cannot be appended, or it cannot be finalized, or the response that finalizes it holds no media or carries no body at all, with the media it initialized, and the error that failed it as the cause



270
271
272
273
274
275
276
277
278
279
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 270

def chunked_upload(media, client:, media_category: nil, media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY,
  shared: nil, additional_owners: nil)
  source = Source.for(media)
  Validator.validate_sharing!(shared, additional_owners)
  Validator.validate_source!(source)
  media_category = Validator.validate_media_category!(media_category || Inference.infer_media_category(source))
  Validator.validate_size!(source, media_category)
  Validator.validate_chunks!(chunk_size:, concurrency:)
  Chunks.upload(client:, source:, media_type: media_type || Inference.infer_media_type(source, media_category), media_category:, chunk_size:, concurrency:, shared:, additional_owners:)
end

.upload(media, client:, media_category: nil, alt_text: nil, processing_timeout: DEFAULT_PROCESSING_TIMEOUT, media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY, shared: nil, additional_owners: nil) ⇒ UploadedMedia

Upload media, in chunks when the API needs them, awaiting any processing

The media is a path, or an IO open on it, which Source says how each of is read.

A video and subtitles upload in chunks, as does an animated GIF that a single request cannot take. Every argument is validated before the first request, so that no media is uploaded, and billed, for an upload that cannot finish.

An image, and a GIF that a single request takes, upload in a single request, which takes no chunks and no media type, so chunk_size, concurrency, and media_type are ignored for them: a chunk_size or a concurrency that is not valid still raises, but none is sent. Media given shared: true uploads in chunks, and uses all three. Upload with #chunked_upload to send an image in chunks too.

Media the response of the upload says is still processing is awaited as #await_processing! awaits it. Media the response says has already failed to process raises MediaProcessingFailed with that response, and media it says has finished is returned as it is, without a check of its status.

The chunks of an upload in chunks are sent by threads of their own, as many as the concurrency, so the on_response of the client runs on those threads for the response of each chunk, and a hook that reads state kept for the thread that called, such as a Rails CurrentAttributes or a logger of its own, reads that of another thread.

Each chunk is a request of its own, which a rate limit can refuse, and a chunk refused raises ChunkedUploadFailed, since the client retries a request refused for a rate limit only max_rate_limit_retries times, which is 0 by default. A large video, uploaded in many chunks, should be uploaded with a client whose max_rate_limit_retries is set, such as X::Client.new(max_rate_limit_retries: 3), so that a rate limit is waited out, up to the max_rate_limit_wait of the client, rather than fail the upload.

Examples:

Upload an image

Uploader::MediaUpload.upload("image.png", client: client)

Upload an image with alt text

Uploader::MediaUpload.upload("cat.jpg", client: client, alt_text: "A cat asleep on a keyboard")

Upload a video and wait until it can be attached to a post

Uploader::MediaUpload.upload("video.mp4", client: client)

Upload an image held in memory, whose category its signature names

Uploader::MediaUpload.upload(StringIO.new(png), client: client)

Upload an image another account may post too

Uploader::MediaUpload.upload("cat.jpg", client: client, additional_owners: [7_505_382])

Parameters:

  • media (String, Pathname, IO, StringIO) —

    the path to the media to upload, or an IO open on it

  • client (Client) —

    the X API client

  • media_category (String, Symbol, nil) (defaults to: nil) —

    the media category, in any case, inferred when nil from the bytes the media begins with, or else from the name of its file

  • alt_text (String, nil) (defaults to: nil) —

    alt text describing the media, for people who cannot see it, of 1 to 1,000 characters

  • processing_timeout (Integer, Float, nil) (defaults to: DEFAULT_PROCESSING_TIMEOUT) —

    the seconds to wait for media, such as a video or an animated GIF, to process, of at least 0, from when it is uploaded, as #await_processing counts them, or nil to wait for as long as processing takes

  • media_type (String, nil) (defaults to: nil) —

    the MIME type of media uploaded in chunks, inferred from the media and category when nil; an upload in a single request sends no type, since the API types the media itself, so one given for an image is not sent

  • chunk_size (Integer, nil) (defaults to: nil) —

    the size of each chunk of media uploaded in chunks, in bytes, of at most 5,242,880, the 5 megabytes the API takes in a segment, or nil for DEFAULT_CHUNK_SIZE, 4,194,304 bytes, or as much more as the media needs to fit the segments the API numbers

  • concurrency (Integer) (defaults to: DEFAULT_CONCURRENCY) —

    the number of chunks uploaded at once, of 1 to MAX_CONCURRENCY

  • shared (Boolean, nil) (defaults to: nil) —

    whether the media is shared, so that it can be sent in more than one direct message, or nil to leave it to the API; media that is shared uploads in chunks, since a single request takes no shared

  • additional_owners (Array<Integer, String>, nil) (defaults to: nil) —

    the identifiers of the users, other than the one who uploads it, who may use the media, or nil for none

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the upload response, or the processing status of media that X processes

Raises:

  • (ArgumentError) —

    if the media is neither a path nor an IO, or is a String that holds a NUL byte or a line break, as the contents of media given in place of its path do

  • (InvalidMedia) —

    if the file does not exist

  • (InvalidMedia) —

    if the media cannot be read, or is empty, which holds nothing to upload

  • (InvalidMedia) —

    if the media is larger than the API takes of its category, whatever the account: 5 megabytes of an image, 15 of a GIF, and one of subtitles, or larger than the 16 gigabytes it takes of any

  • (ArgumentError) —

    if the media category is invalid, the alt text is empty or longer than the API takes, the chunk size is not a positive Integer, is larger than a segment the API takes, or would need more segments than the API numbers, the concurrency is not 1 to MAX_CONCURRENCY, the processing timeout is neither nil nor a finite number of seconds of at least 0, shared is neither true, false, nor nil, or additional_owners is neither nil nor an Array of at least one user identifier

  • (InvalidMediaType) —

    if no media category is given for media whose type neither its bytes nor the name of its file names, if media uploaded in chunks is given no media type and none can be inferred, if the category does not take the type of the media, such as an MP4 video uploaded as a GIF, or if the file is named as a type every file of which begins with a signature, such as a PNG, and does not begin with it

  • (MissingMediaData) —

    if a response of the upload holds no media, or carries no body at all

  • (ChunkedUploadFailed) —

    if media uploaded in chunks is initialized, but a chunk cannot be appended, or it cannot be finalized, with the media it initialized

  • (MediaProcessingFailed) —

    if the media fails to process, or its processing ends in no state X documents

  • (MediaProcessingTimeout) —

    if the media is still processing once processing_timeout seconds would pass

  • (MediaProcessingCheckFailed) —

    if the media is uploaded, but a check of its processing fails, as when the API answers it with an error, with the media it uploaded

  • (AltTextFailed) —

    if the media is uploaded, but its alt text cannot be added, with the media it uploaded



205
206
207
208
209
210
211
212
213
214
215
216
217
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 205

def upload(media, client:, media_category: nil, alt_text: nil, processing_timeout: DEFAULT_PROCESSING_TIMEOUT,
  media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY, shared: nil, additional_owners: nil)
  source = Source.for(media)
  media_category = Validator.validate_upload!(source, media_category, alt_text:, chunk_size:, concurrency:, processing_timeout:, shared:, additional_owners:) { Inference.infer_media_category(source) }
  uploaded = if shared || Inference.chunked_upload?(source, media_category)
    Chunks.upload(client:, source:, media_type: media_type || Inference.infer_media_type(source, media_category), media_category:, chunk_size:, concurrency:, shared:, additional_owners:)
  else
    Utils.single_request(client, Inference.single_request!(source, media_category), media_category, additional_owners:)
  end
  uploaded = Utils.processed!(uploaded.processing? ? MediaProcessingCheckFailed.__send__(:keeping, uploaded) { MediaUpload.await_processing(uploaded, client:, processing_timeout:) } : uploaded)
  AltTextFailed.__send__(:keeping, uploaded) { Metadata.add_alt_text(uploaded, alt_text, client:) } unless alt_text.nil?
  uploaded
end

Instance Method Details

#await_processing(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT) ⇒ UploadedMedia

Wait for media processing to complete

Media that already says its processing has ended, in success or failure, or that holds an upload response which names no processing, as that of an image does, is returned as it is, without a request, since a check would tell no more than the media holds; await_processing! raises for media that says it failed, without a request too.

Before each check it waits as long as X asked, and at least a second: media an upload returned, or the Hash of its response, which says how long to wait before the first check, is not checked until then, and media given as its identifier, or its identifier and media key alone, which say nothing of its processing, or as media in a state of processing X does not document, is checked at once.

The processing timeout is a deadline, the seconds from when it is called, measured on the monotonic clock, so that it counts the time each check takes, with any wait for a rate limit and any retry the client makes, as well as the waits between them. It gives up once the next check X asks for would come after the deadline, rather than sleep past it, or check before X asks. A check under way at the deadline is let finish, and its status returned if processing has finished, so it can return that much after the deadline.

It waits while the processing is pending or in progress alone, so a status whose processing names no state, or a state X does not document, is returned as it is, neither processing nor ready, rather than checked until the deadline, since X gives no time to check it again at; await_processing! raises for it.

Examples:

Wait for processing

Uploader::MediaUpload.await_processing(media, client: client)

Wait for the processing of media known by its identifier

Uploader::MediaUpload.await_processing("1880028106020515840", client: client)

Wait up to half an hour for a long video

Uploader::MediaUpload.await_processing(media, client: client, processing_timeout: 1800)

Parameters:

  • media (UploadedMedia, Hash, #media_key, String, Integer) —

    the uploaded media, media that has a media key, such as X::Media, the media key, or the media identifier

  • client (Client) —

    the X API client

  • processing_timeout (Integer, Float, nil) (defaults to: DEFAULT_PROCESSING_TIMEOUT) —

    the seconds from now to wait for processing to finish, checks and all, before giving up, or nil to wait for as long as processing takes

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the processing status, or the media given, as uploaded media, if its processing has already ended

Raises:

  • (ArgumentError) —

    if the processing timeout is neither a finite number of seconds of at least 0 nor nil

  • (ArgumentError) —

    if the media given is nil, holds no identifier, or is neither media, a media key, nor a media identifier, or its media key names none

  • (MissingMediaData) —

    if a status response holds no media or carries no body at all

  • (MediaProcessingTimeout) —

    if the media is still processing once the next check would pass the deadline



322
323
324
325
326
327
328
329
330
331
332
333
334
335
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 322

def await_processing(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT)
  Validator.validate_processing_timeout!(processing_timeout)
  uploaded = Utils.uploaded_media(media)
  return uploaded if uploaded.ready? || uploaded.failed?

  deadline, media_id, pending = processing_timeout&.then { |seconds| Utils.seconds_from_now(seconds) }, Utils.media_id(uploaded), (uploaded if uploaded.processing?)
  loop do
    Utils.wait_to_check(pending, deadline:, timeout: processing_timeout) if pending
    status = UploadedMedia.new(Utils.media_data(client.get("media/upload", params: {command: STATUS_COMMAND, media_id:}, **JSON_CLASSES), "of the status check"))
    return status unless status.processing?

    pending = status
  end
end

#await_processing!(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT) ⇒ UploadedMedia

Wait for media processing and raise on failure

Examples:

Wait for processing with error handling

Uploader::MediaUpload.await_processing!(media, client: client)

Parameters:

  • media (UploadedMedia, Hash, #media_key, String, Integer) —

    the uploaded media, media that has a media key, such as X::Media, the media key, or the media identifier

  • client (Client) —

    the X API client

  • processing_timeout (Integer, Float, nil) (defaults to: DEFAULT_PROCESSING_TIMEOUT) —

    the seconds from now to wait for processing to finish, checks and all, before giving up, as #await_processing counts them, or nil to wait for as long as processing takes

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the processing status, or the media given, as uploaded media, if its processing has already succeeded

Raises:

  • (ArgumentError) —

    if the processing timeout is neither a finite number of seconds of at least 0 nor nil

  • (ArgumentError) —

    if the media given is nil, holds no identifier, or is neither media, a media key, nor a media identifier, or its media key names none

  • (MissingMediaData) —

    if a status response holds no media or carries no body at all

  • (MediaProcessingFailed) —

    if media processing failed, or ended in no state X documents, with the status X reported, or the media given, without a request, if it already says its processing failed

  • (MediaProcessingTimeout) —

    if the media is still processing once the next check would pass the deadline



356
357
358
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 356

def await_processing!(media, client:, processing_timeout: DEFAULT_PROCESSING_TIMEOUT)
  Utils.processed!(MediaUpload.await_processing(media, client:, processing_timeout:))
end

#chunked_upload(media, client:, media_category: nil, media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY, shared: nil, additional_owners: nil) ⇒ UploadedMedia

Perform a chunked upload for large files

It is the way to upload media without waiting for X to process it: #upload waits for the processing of media X processes, such as a video, and this returns once the upload is finalized, so that the caller can go on while X processes a long video, and wait for it with #await_processing or #await_processing! when it needs it. It uploads in chunks whatever the media, an image as well.

The chunks of an upload in chunks are sent by threads of their own, as many as the concurrency, so the on_response of the client runs on those threads for the response of each chunk, and a hook that reads state kept for the thread that called, such as a Rails CurrentAttributes or a logger of its own, reads that of another thread.

Each chunk is a request of its own, which a rate limit can refuse, and a chunk refused raises ChunkedUploadFailed, since the client retries a request refused for a rate limit only max_rate_limit_retries times, which is 0 by default. A large video, uploaded in many chunks, should be uploaded with a client whose max_rate_limit_retries is set, such as X::Client.new(max_rate_limit_retries: 3), so that a rate limit is waited out, up to the max_rate_limit_wait of the client, rather than fail the upload.

Examples:

Upload a large video

Uploader::MediaUpload.chunked_upload("video.mp4", client: client)

Parameters:

  • media (String, Pathname, IO, StringIO) —

    the path to the media to upload, or an IO open on it

  • client (Client) —

    the X API client

  • media_category (String, Symbol, nil) (defaults to: nil) —

    the media category, in any case, inferred from the media when nil

  • media_type (String, nil) (defaults to: nil) —

    the MIME type of the media, sent as it is given, or inferred from the media and category when nil

  • chunk_size (Integer, nil) (defaults to: nil) —

    the size of each chunk in bytes, of at most 5,242,880, the 5 megabytes the API takes in a segment, or nil for DEFAULT_CHUNK_SIZE, 4,194,304 bytes, or as much more as the media needs to fit the segments the API numbers

  • concurrency (Integer) (defaults to: DEFAULT_CONCURRENCY) —

    the number of chunks uploaded at once, of 1 to MAX_CONCURRENCY

  • shared (Boolean, nil) (defaults to: nil) —

    whether the media is shared, so that it can be sent in more than one direct message, or nil to leave it to the API

  • additional_owners (Array<Integer, String>, nil) (defaults to: nil) —

    the identifiers of the users, other than the one who uploads it, who may use the media, or nil for none

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the upload response

Raises:

  • (ArgumentError) —

    if the media is neither a path nor an IO, or is a String that holds a NUL byte or a line break, as the contents of media given in place of its path do

  • (InvalidMedia) —

    if the file does not exist

  • (InvalidMedia) —

    if the media cannot be read, or is empty, which holds nothing to upload

  • (InvalidMedia) —

    if the media is larger than the API takes of its category, whatever the account: 5 megabytes of an image, 15 of a GIF, and one of subtitles, or larger than the 16 gigabytes it takes of any

  • (ArgumentError) —

    if the media category is invalid, the chunk size is not a positive Integer, is larger than a segment the API takes, or would need more segments than the API numbers, the concurrency is not 1 to MAX_CONCURRENCY, shared is neither true, false, nor nil, or additional_owners is neither nil nor an Array of at least one user identifier

  • (InvalidMediaType) —

    if no media type is given and none can be inferred, or the one the media is, read from its bytes or else from the name of its file, is not one the category takes

  • (MissingMediaData) —

    if the response that initializes the upload holds no media to append the chunks to

  • (ChunkedUploadFailed) —

    if the upload is initialized, but a chunk cannot be appended, or it cannot be finalized, or the response that finalizes it holds no media or carries no body at all, with the media it initialized, and the error that failed it as the cause



270
271
272
273
274
275
276
277
278
279
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 270

def chunked_upload(media, client:, media_category: nil, media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY,
  shared: nil, additional_owners: nil)
  source = Source.for(media)
  Validator.validate_sharing!(shared, additional_owners)
  Validator.validate_source!(source)
  media_category = Validator.validate_media_category!(media_category || Inference.infer_media_category(source))
  Validator.validate_size!(source, media_category)
  Validator.validate_chunks!(chunk_size:, concurrency:)
  Chunks.upload(client:, source:, media_type: media_type || Inference.infer_media_type(source, media_category), media_category:, chunk_size:, concurrency:, shared:, additional_owners:)
end

#upload(media, client:, media_category: nil, alt_text: nil, processing_timeout: DEFAULT_PROCESSING_TIMEOUT, media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY, shared: nil, additional_owners: nil) ⇒ UploadedMedia

Upload media, in chunks when the API needs them, awaiting any processing

The media is a path, or an IO open on it, which Source says how each of is read.

A video and subtitles upload in chunks, as does an animated GIF that a single request cannot take. Every argument is validated before the first request, so that no media is uploaded, and billed, for an upload that cannot finish.

An image, and a GIF that a single request takes, upload in a single request, which takes no chunks and no media type, so chunk_size, concurrency, and media_type are ignored for them: a chunk_size or a concurrency that is not valid still raises, but none is sent. Media given shared: true uploads in chunks, and uses all three. Upload with #chunked_upload to send an image in chunks too.

Media the response of the upload says is still processing is awaited as #await_processing! awaits it. Media the response says has already failed to process raises MediaProcessingFailed with that response, and media it says has finished is returned as it is, without a check of its status.

The chunks of an upload in chunks are sent by threads of their own, as many as the concurrency, so the on_response of the client runs on those threads for the response of each chunk, and a hook that reads state kept for the thread that called, such as a Rails CurrentAttributes or a logger of its own, reads that of another thread.

Each chunk is a request of its own, which a rate limit can refuse, and a chunk refused raises ChunkedUploadFailed, since the client retries a request refused for a rate limit only max_rate_limit_retries times, which is 0 by default. A large video, uploaded in many chunks, should be uploaded with a client whose max_rate_limit_retries is set, such as X::Client.new(max_rate_limit_retries: 3), so that a rate limit is waited out, up to the max_rate_limit_wait of the client, rather than fail the upload.

Examples:

Upload an image

Uploader::MediaUpload.upload("image.png", client: client)

Upload an image with alt text

Uploader::MediaUpload.upload("cat.jpg", client: client, alt_text: "A cat asleep on a keyboard")

Upload a video and wait until it can be attached to a post

Uploader::MediaUpload.upload("video.mp4", client: client)

Upload an image held in memory, whose category its signature names

Uploader::MediaUpload.upload(StringIO.new(png), client: client)

Upload an image another account may post too

Uploader::MediaUpload.upload("cat.jpg", client: client, additional_owners: [7_505_382])

Parameters:

  • media (String, Pathname, IO, StringIO) —

    the path to the media to upload, or an IO open on it

  • client (Client) —

    the X API client

  • media_category (String, Symbol, nil) (defaults to: nil) —

    the media category, in any case, inferred when nil from the bytes the media begins with, or else from the name of its file

  • alt_text (String, nil) (defaults to: nil) —

    alt text describing the media, for people who cannot see it, of 1 to 1,000 characters

  • processing_timeout (Integer, Float, nil) (defaults to: DEFAULT_PROCESSING_TIMEOUT) —

    the seconds to wait for media, such as a video or an animated GIF, to process, of at least 0, from when it is uploaded, as #await_processing counts them, or nil to wait for as long as processing takes

  • media_type (String, nil) (defaults to: nil) —

    the MIME type of media uploaded in chunks, inferred from the media and category when nil; an upload in a single request sends no type, since the API types the media itself, so one given for an image is not sent

  • chunk_size (Integer, nil) (defaults to: nil) —

    the size of each chunk of media uploaded in chunks, in bytes, of at most 5,242,880, the 5 megabytes the API takes in a segment, or nil for DEFAULT_CHUNK_SIZE, 4,194,304 bytes, or as much more as the media needs to fit the segments the API numbers

  • concurrency (Integer) (defaults to: DEFAULT_CONCURRENCY) —

    the number of chunks uploaded at once, of 1 to MAX_CONCURRENCY

  • shared (Boolean, nil) (defaults to: nil) —

    whether the media is shared, so that it can be sent in more than one direct message, or nil to leave it to the API; media that is shared uploads in chunks, since a single request takes no shared

  • additional_owners (Array<Integer, String>, nil) (defaults to: nil) —

    the identifiers of the users, other than the one who uploads it, who may use the media, or nil for none

Returns:

  • (UploadedMedia) —

    the uploaded media, which holds the upload response, or the processing status of media that X processes

Raises:

  • (ArgumentError) —

    if the media is neither a path nor an IO, or is a String that holds a NUL byte or a line break, as the contents of media given in place of its path do

  • (InvalidMedia) —

    if the file does not exist

  • (InvalidMedia) —

    if the media cannot be read, or is empty, which holds nothing to upload

  • (InvalidMedia) —

    if the media is larger than the API takes of its category, whatever the account: 5 megabytes of an image, 15 of a GIF, and one of subtitles, or larger than the 16 gigabytes it takes of any

  • (ArgumentError) —

    if the media category is invalid, the alt text is empty or longer than the API takes, the chunk size is not a positive Integer, is larger than a segment the API takes, or would need more segments than the API numbers, the concurrency is not 1 to MAX_CONCURRENCY, the processing timeout is neither nil nor a finite number of seconds of at least 0, shared is neither true, false, nor nil, or additional_owners is neither nil nor an Array of at least one user identifier

  • (InvalidMediaType) —

    if no media category is given for media whose type neither its bytes nor the name of its file names, if media uploaded in chunks is given no media type and none can be inferred, if the category does not take the type of the media, such as an MP4 video uploaded as a GIF, or if the file is named as a type every file of which begins with a signature, such as a PNG, and does not begin with it

  • (MissingMediaData) —

    if a response of the upload holds no media, or carries no body at all

  • (ChunkedUploadFailed) —

    if media uploaded in chunks is initialized, but a chunk cannot be appended, or it cannot be finalized, with the media it initialized

  • (MediaProcessingFailed) —

    if the media fails to process, or its processing ends in no state X documents

  • (MediaProcessingTimeout) —

    if the media is still processing once processing_timeout seconds would pass

  • (MediaProcessingCheckFailed) —

    if the media is uploaded, but a check of its processing fails, as when the API answers it with an error, with the media it uploaded

  • (AltTextFailed) —

    if the media is uploaded, but its alt text cannot be added, with the media it uploaded



205
206
207
208
209
210
211
212
213
214
215
216
217
 
# File 'x-uploader/lib/x/uploader/media_upload.rb', line 205

def upload(media, client:, media_category: nil, alt_text: nil, processing_timeout: DEFAULT_PROCESSING_TIMEOUT,
  media_type: nil, chunk_size: nil, concurrency: DEFAULT_CONCURRENCY, shared: nil, additional_owners: nil)
  source = Source.for(media)
  media_category = Validator.validate_upload!(source, media_category, alt_text:, chunk_size:, concurrency:, processing_timeout:, shared:, additional_owners:) { Inference.infer_media_category(source) }
  uploaded = if shared || Inference.chunked_upload?(source, media_category)
    Chunks.upload(client:, source:, media_type: media_type || Inference.infer_media_type(source, media_category), media_category:, chunk_size:, concurrency:, shared:, additional_owners:)
  else
    Utils.single_request(client, Inference.single_request!(source, media_category), media_category, additional_owners:)
  end
  uploaded = Utils.processed!(uploaded.processing? ? MediaProcessingCheckFailed.__send__(:keeping, uploaded) { MediaUpload.await_processing(uploaded, client:, processing_timeout:) } : uploaded)
  AltTextFailed.__send__(:keeping, uploaded) { Metadata.add_alt_text(uploaded, alt_text, client:) } unless alt_text.nil?
  uploaded
end