List of usage examples for com.amazonaws.services.sqs.model SendMessageBatchRequest getEntries
public java.util.List<SendMessageBatchRequestEntry> getEntries()
A list of SendMessageBatchRequestEntry items.
From source file:com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.java
License:Open Source License
/** * <p>//w w w. ja v a 2 s .c o m * Delivers up to ten messages to the specified queue. This is a batch * version of SendMessage. The result of the send action on each message is * reported individually in the response. Uploads message payloads to Amazon * S3 when necessary. * </p> * <p> * If the <code>DelaySeconds</code> parameter is not specified for an entry, * the default for the queue is used. * </p> * <p> * <b>IMPORTANT:</b>The following list shows the characters (in Unicode) * that are allowed in your message, according to the W3C XML specification. * For more information, go to http://www.faqs.org/rfcs/rfc1321.html. If you * send any characters that are not included in the list, your request will * be rejected. #x9 | #xA | #xD | [#x20 to #xD7FF] | [#xE000 to #xFFFD] | * [#x10000 to #x10FFFF] * </p> * <p> * <b>IMPORTANT:</b> Because the batch request can result in a combination * of successful and unsuccessful actions, you should check for batch errors * even when the call returns an HTTP status code of 200. * </p> * <b>IMPORTANT:</b> The input object may be modified by the method. </p> * <p> * <b>NOTE:</b>Some API actions take lists of parameters. These lists are * specified using the param.n notation. Values of n are integers starting * from 1. For example, a parameter list with two elements looks like this: * </p> * <p> * <code>&Attribute.1=this</code> * </p> * <p> * <code>&Attribute.2=that</code> * </p> * * @param sendMessageBatchRequest * Container for the necessary parameters to execute the * SendMessageBatch service method on AmazonSQS. * * @return The response from the SendMessageBatch service method, as * returned by AmazonSQS. * * @throws BatchEntryIdsNotDistinctException * @throws TooManyEntriesInBatchRequestException * @throws BatchRequestTooLongException * @throws UnsupportedOperationException * @throws InvalidBatchEntryIdException * @throws EmptyBatchRequestException * * @throws AmazonClientException * If any internal errors are encountered inside the client * while attempting to make the request or handle the response. * For example if a network connection is not available. * @throws AmazonServiceException * If an error response is returned by AmazonSQS indicating * either a problem with the data in the request, or a server * side issue. */ public SendMessageBatchResult sendMessageBatch(SendMessageBatchRequest sendMessageBatchRequest) { if (sendMessageBatchRequest == null) { String errorMessage = "sendMessageBatchRequest cannot be null."; LOG.error(errorMessage); throw new AmazonClientException(errorMessage); } sendMessageBatchRequest.getRequestClientOptions() .appendUserAgent(SQSExtendedClientConstants.USER_AGENT_HEADER); if (!clientConfiguration.isLargePayloadSupportEnabled()) { return super.sendMessageBatch(sendMessageBatchRequest); } List<SendMessageBatchRequestEntry> batchEntries = sendMessageBatchRequest.getEntries(); int index = 0; for (SendMessageBatchRequestEntry entry : batchEntries) { if (clientConfiguration.isAlwaysThroughS3() || isLarge(entry)) { batchEntries.set(index, storeMessageInS3(entry)); } ++index; } return super.sendMessageBatch(sendMessageBatchRequest); }
From source file:com.dushyant.flume.sink.aws.sqs.BatchSQSMsgSender.java
License:Apache License
/** * Handles SQS send message batch result and throws EventDeliveryException to cause the flume transaction to fail * and let flume retry the whole batch in case all the messages in the batch failed to be delivered to SQS. * Currently, this method does just logs errors and skips the messages in case some messages from the batched failed * to be delivered but some succeeded (i.e., partial batch failure). * <p>//from w w w.jav a 2 s. co m * TODO: Add retry logic instead letting flume drop the failed messages in case of partial batch failure * * @param batchRequest The SQS SendMessageBatchRequest * @param batchResult The SQS SendMessageBatchResult * * @throws EventDeliveryException In case all the messages in the batch failed to be delivered to SQS */ protected void handleResult(SendMessageBatchRequest batchRequest, SendMessageBatchResult batchResult) throws EventDeliveryException { List<SendMessageBatchRequestEntry> batchRequestEntries = batchRequest.getEntries(); List<BatchResultErrorEntry> errors = batchResult.getFailed(); int attemptedCount = batchRequestEntries == null ? 0 : batchRequestEntries.size(); int errorCount = errors == null ? 0 : errors.size(); if (errorCount > 0) { String errorMessage = buildErrorMessage(batchRequestEntries, errors); if (attemptedCount == errorCount) { // if it was a non-empty batch and if all the messages in the batch have errors then fail the whole // batch and let flume rollback the transaction and retry it // Just throw the EventDeliveryException. This will eventually cause the channel's transaction to // rollback. throw new EventDeliveryException(errorMessage); } else { // TODO: Add retry logic instead letting flume drop the failed messages in case of partial batch failure // Just log the error message and let flume drop failed messages in case of partial batch failures LOG.error(errorMessage); } } }
From source file:com.dushyant.flume.sink.aws.sqs.BatchSQSMsgSender.java
License:Apache License
/** * The method creates batch of requests based on the configured "batchSize" to send to SQS. * <p>/*w w w. ja v a2 s.c om*/ * When the combined payload size of the messages in the batch increases beyond the configured max allowed limit * then the method rolls the remaining messages into another batch. * <p> * For example, let's say that the <i>batchSize</i> is 10 and <i>maxMessageSize</i> is 256Kb and after creating a * batch with 7 messages the 256KB size limit is reached, in that case the method will split the remaining 3(i.e. 10 * - 7) messages into its own batch request. * <p> * The returned list from this method will usually contain only 1 batch request (with all the messages part of that * batch request) when the total message size in the batch is within the allowed limit. * * @param channel Flume channel to take the messages from * * @return A list of {@link SendMessageBatchRequest} objects. * * @throws EventDeliveryException In case the message to be sent to SQS cannot be encoded in UTF-8 format */ protected List<SendMessageBatchRequest> createBatches(Channel channel) throws EventDeliveryException { List<SendMessageBatchRequest> batchRequests = new ArrayList<SendMessageBatchRequest>(); // Create a batch request SendMessageBatchRequest batchRequest = new SendMessageBatchRequest(sqsUrl); Collection<SendMessageBatchRequestEntry> entries = new ArrayList<SendMessageBatchRequestEntry>(); long numberOfBytesInBatch = 0; for (int i = 0; i < batchSize; ++i) { // Take event from the channel and add corresponding message to the batch Event event = channel.take(); byte[] msgBytes = (event == null) ? null : event.getBody(); if (msgBytes == null || msgBytes.length == 0) { // Channel returned null or empty event. Just break. Create batch with whatever events we have got so // far. This can happen when the channel is empty or is receiving events but with empty body (For // example, exec tail source may send empty events). break; } // Using message number as Id. This id is used for identifying the message entries within the batch. // It needs to be unique within a batch. String id = String.valueOf(i + 1); numberOfBytesInBatch += msgBytes.length; if (numberOfBytesInBatch > maxMessageSize) { // Max size per batch reached. Split into another batch. // Add entries collected so far into the current batch batchRequest.setEntries(entries); // Add the current batch into the list of batches to return batchRequests.add(batchRequest); // reset byte counter numberOfBytesInBatch = 0; // create new batch request batchRequest = new SendMessageBatchRequest(sqsUrl); // reset entries for the new batch entries = new ArrayList<SendMessageBatchRequestEntry>(); } SendMessageBatchRequestEntry requestEntry = null; try { requestEntry = new SendMessageBatchRequestEntry(id, new String(msgBytes, "UTF-8").trim()); } catch (UnsupportedEncodingException e) { throw new EventDeliveryException("Character set UTF-8 not supported.", e); } entries.add(requestEntry); } // The last batch request may not have been populated yet. Populate if there are any entries to be populated. if ((batchRequest.getEntries() == null || batchRequest.getEntries().size() == 0) && entries.size() > 0) { batchRequest.setEntries(entries); batchRequests.add(batchRequest); } return batchRequests; }
From source file:com.netflix.conductor.contribs.queue.sqs.SQSObservableQueue.java
License:Apache License
void publishMessages(List<Message> messages) { logger.info("Sending {} messages", messages.size()); SendMessageBatchRequest batch = new SendMessageBatchRequest(queueURL); messages.stream().forEach(msg -> { SendMessageBatchRequestEntry sendr = new SendMessageBatchRequestEntry(msg.getId(), msg.getPayload()); batch.getEntries().add(sendr); });//from ww w . j a v a 2 s . co m logger.info("sending {}", batch.getEntries().size()); SendMessageBatchResult result = client.sendMessageBatch(batch); logger.info("send result {}", result.getFailed().toString()); }
From source file:smartthings.brave.sqs.TracingAmazonSQSClient.java
License:Apache License
@Override public SendMessageBatchResult sendMessageBatch(SendMessageBatchRequest sendMessageBatchRequest) { List<Span> oneWays = new LinkedList<>(); for (SendMessageBatchRequestEntry entry : sendMessageBatchRequest.getEntries()) { Span s = withEndpoint(tracing.tracer().nextSpan()).kind(Span.Kind.CLIENT).start(); parser.request(sendMessageBatchRequest, s); injector.inject(s.context(), entry.getMessageAttributes()); oneWays.add(s);// w ww .j a va2s . c o m } SendMessageBatchResult result = delegate.sendMessageBatch(sendMessageBatchRequest); // flush after success so we don't start one way spans on a request failure. for (Span oneWay : oneWays) { oneWay.flush(); } return result; }