Example usage for com.amazonaws.services.sqs.model Message getMessageAttributes

List of usage examples for com.amazonaws.services.sqs.model Message getMessageAttributes

  1. HOME
  2. Java
  3. com.amazonaws
  4. com.amazonaws.services.sqs.model.*
  5. Message
  6. getMessageAttributes

Introduction

In this page you can find the example usage for com.amazonaws.services.sqs.model Message getMessageAttributes.

Prototype


public java.util.Map<String, MessageAttributeValue> getMessageAttributes()

Source Link

Document

Each message attribute consists of a Name, Type, and Value.

Usage

From source file:com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.java

License:Open Source License

/**
 * <p>// w  ww  .j a v  a 2 s .  c  om
 * Retrieves one or more messages, with a maximum limit of 10 messages, from
 * the specified queue. Downloads the message payloads from Amazon S3 when
 * necessary. Long poll support is enabled by using the
 * <code>WaitTimeSeconds</code> parameter. For more information, see <a
 * href=
 * "http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html"
 * > Amazon SQS Long Poll </a> in the <i>Amazon SQS Developer Guide</i> .
 * </p>
 * <p>
 * Short poll is the default behavior where a weighted random set of
 * machines is sampled on a <code>ReceiveMessage</code> call. This means
 * only the messages on the sampled machines are returned. If the number of
 * messages in the queue is small (less than 1000), it is likely you will
 * get fewer messages than you requested per <code>ReceiveMessage</code>
 * call. If the number of messages in the queue is extremely small, you
 * might not receive any messages in a particular
 * <code>ReceiveMessage</code> response; in which case you should repeat the
 * request.
 * </p>
 * <p>
 * For each message returned, the response includes the following:
 * </p>
 * 
 * <ul>
 * <li>
 * <p>
 * Message body
 * </p>
 * </li>
 * <li>
 * <p>
 * MD5 digest of the message body. For information about MD5, go to <a
 * href="http://www.faqs.org/rfcs/rfc1321.html">
 * http://www.faqs.org/rfcs/rfc1321.html </a> .
 * </p>
 * </li>
 * <li>
 * <p>
 * Message ID you received when you sent the message to the queue.
 * </p>
 * </li>
 * <li>
 * <p>
 * Receipt handle.
 * </p>
 * </li>
 * <li>
 * <p>
 * Message attributes.
 * </p>
 * </li>
 * <li>
 * <p>
 * MD5 digest of the message attributes.
 * </p>
 * </li>
 * 
 * </ul>
 * <p>
 * The receipt handle is the identifier you must provide when deleting the
 * message. For more information, see <a href=
 * "http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/ImportantIdentifiers.html"
 * > Queue and Message Identifiers </a> in the <i>Amazon SQS Developer
 * Guide</i> .
 * </p>
 * <p>
 * You can provide the <code>VisibilityTimeout</code> parameter in your
 * request, which will be applied to the messages that Amazon SQS returns in
 * the response. If you do not include the parameter, the overall visibility
 * timeout for the queue is used for the returned messages. For more
 * information, see <a href=
 * "http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AboutVT.html"
 * > Visibility Timeout </a> in the <i>Amazon SQS Developer Guide</i> .
 * </p>
 * <p>
 * <b>NOTE:</b> Going forward, new attributes might be added. If you are
 * writing code that calls this action, we recommend that you structure your
 * code so that it can handle new attributes gracefully.
 * </p>
 *
 * @param receiveMessageRequest
 *            Container for the necessary parameters to execute the
 *            ReceiveMessage service method on AmazonSQS.
 * 
 * @return The response from the ReceiveMessage service method, as returned
 *         by AmazonSQS.
 * 
 * @throws OverLimitException
 *
 * @throws AmazonClientException
 *             If any internal errors are encountered inside the client
 *             while attempting to make the request or handle the response.
 *             For example if a network connection is not available.
 * @throws AmazonServiceException
 *             If an error response is returned by AmazonSQS indicating
 *             either a problem with the data in the request, or a server
 *             side issue.
 */
public ReceiveMessageResult receiveMessage(ReceiveMessageRequest receiveMessageRequest) {

    if (receiveMessageRequest == null) {
        String errorMessage = "receiveMessageRequest cannot be null.";
        LOG.error(errorMessage);
        throw new AmazonClientException(errorMessage);
    }

    receiveMessageRequest.getRequestClientOptions()
            .appendUserAgent(SQSExtendedClientConstants.USER_AGENT_HEADER);

    if (!clientConfiguration.isLargePayloadSupportEnabled()) {
        return super.receiveMessage(receiveMessageRequest);
    }

    receiveMessageRequest.getMessageAttributeNames().add(SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME);

    ReceiveMessageResult receiveMessageResult = super.receiveMessage(receiveMessageRequest);

    List<Message> messages = receiveMessageResult.getMessages();
    for (Message message : messages) {

        // for each received message check if they are stored in S3.
        MessageAttributeValue largePayloadAttributeValue = message.getMessageAttributes()
                .get(SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME);
        if (largePayloadAttributeValue != null) {
            String messageBody = message.getBody();

            // read the S3 pointer from the message body JSON string.
            MessageS3Pointer s3Pointer = readMessageS3PointerFromJSON(messageBody);

            String s3MsgBucketName = s3Pointer.getS3BucketName();
            String s3MsgKey = s3Pointer.getS3Key();

            String origMsgBody = getTextFromS3(s3MsgBucketName, s3MsgKey);
            LOG.info("S3 object read, Bucket name: " + s3MsgBucketName + ", Object key: " + s3MsgKey + ".");

            message.setBody(origMsgBody);

            // remove the additional attribute before returning the message
            // to user.
            message.getMessageAttributes().remove(SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME);

            // Embed s3 object pointer in the receipt handle.
            String modifiedReceiptHandle = embedS3PointerInReceiptHandle(message.getReceiptHandle(),
                    s3MsgBucketName, s3MsgKey);

            message.setReceiptHandle(modifiedReceiptHandle);
        }
    }
    return receiveMessageResult;
}

From source file:com.amazon.sqs.javamessaging.SQSMessageConsumerPrefetch.java

License:Open Source License

/**
 * Convert the return SQS message into JMS message
 * @param message SQS message to convert
 * @return Converted JMS message/*  w w w . ja v a 2  s  .  c o m*/
 * @throws JMSException
 */
protected javax.jms.Message convertToJMSMessage(Message message) throws JMSException {
    MessageAttributeValue messageTypeAttribute = message.getMessageAttributes()
            .get(SQSMessage.JMS_SQS_MESSAGE_TYPE);
    javax.jms.Message jmsMessage = null;
    if (messageTypeAttribute == null) {
        jmsMessage = new SQSTextMessage(acknowledger, queueUrl, message);
    } else {
        String messageType = messageTypeAttribute.getStringValue();
        if (SQSMessage.BYTE_MESSAGE_TYPE.equals(messageType)) {
            try {
                jmsMessage = new SQSBytesMessage(acknowledger, queueUrl, message);
            } catch (JMSException e) {
                LOG.warn("MessageReceiptHandle - " + message.getReceiptHandle()
                        + "cannot be serialized to BytesMessage", e);
                throw e;
            }
        } else if (SQSMessage.OBJECT_MESSAGE_TYPE.equals(messageType)) {
            jmsMessage = new SQSObjectMessage(acknowledger, queueUrl, message);
        } else if (SQSMessage.TEXT_MESSAGE_TYPE.equals(messageType)) {
            jmsMessage = new SQSTextMessage(acknowledger, queueUrl, message);
        } else {
            throw new JMSException("Not a supported JMS message type");
        }
    }
    jmsMessage.setJMSDestination(sqsDestination);
    return jmsMessage;
}

From source file:com.mateusz.mateuszsqs.SQSConfig.java

public static List<String> getMessages(AmazonSQS sqs, String sqsURL) {
    ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(sqsURL);
    List<Message> messages = sqs.receiveMessage(receiveMessageRequest.withMessageAttributeNames("All"))
            .getMessages();/* w w w .  j  av a  2 s .  c o m*/
    List<String> filesToProcess = new ArrayList<String>();
    for (Message message : messages) {
        System.out.println("  Message");
        System.out.println("    MessageId:     " + message.getMessageId());
        System.out.println("    ReceiptHandle: " + message.getReceiptHandle());
        System.out.println("    MD5OfBody:     " + message.getMD5OfBody());
        System.out.println("    Body:          " + message.getBody());
        for (Map.Entry<String, MessageAttributeValue> entry : message.getMessageAttributes().entrySet()) {
            System.out.println("  Attribute");
            System.out.println("    Name:  " + entry.getKey());
            System.out.println("    Value: " + entry.getValue().getStringValue());
            filesToProcess.add(entry.getValue().getStringValue());
        }
        System.out.println("Deleting a message.\n");
        String messageReceiptHandle = message.getReceiptHandle();
        sqs.deleteMessage(new DeleteMessageRequest(sqsURL, messageReceiptHandle));
    }

    return filesToProcess;
}

From source file:com.netflix.spinnaker.echo.pubsub.amazon.SQSSubscriber.java

License:Apache License

private void handleMessage(Message message) {
    try {//from   www .j a v a2 s.c om
        String messageId = message.getMessageId();
        String messagePayload = unmarshallMessageBody(message.getBody());

        Map<String, String> stringifiedMessageAttributes = message.getMessageAttributes().entrySet().stream()
                .collect(Collectors.toMap(Map.Entry::getKey, e -> String.valueOf(e.getValue())));

        log.debug("Received Amazon sqs message: {} with payload: {} and attributes: {}", messageId,
                messagePayload, stringifiedMessageAttributes);

        MessageDescription description = MessageDescription.builder().subscriptionName(subscriptionName())
                .messagePayload(messagePayload).messageAttributes(stringifiedMessageAttributes)
                .pubsubSystem(pubsubSystem).ackDeadlineMillis(TimeUnit.SECONDS.toMillis(50)) // Set a high upper bound on message processing time.
                .retentionDeadlineMillis(TimeUnit.DAYS.toMillis(7)) // Expire key after max retention time, which is 7 days.
                .build();

        AmazonMessageAcknowledger acknowledger = new AmazonMessageAcknowledger(amazonSQS, queueId, message,
                registry, getName());

        if (subscription.getMessageFormat() != AmazonPubsubProperties.MessageFormat.NONE) {
            description.setArtifacts(parseArtifacts(description.getMessagePayload(), messageId));
        }
        pubsubMessageHandler.handleMessage(description, acknowledger, identity.getIdentity(), messageId);
    } catch (Exception e) {
        log.error("Message {} from queue {} failed to be handled", message, queueId);
        // Todo emjburns: add dead-letter queue policy
    }
}

From source file:com.netflix.spinnaker.echo.pubsub.aws.SQSSubscriber.java

License:Apache License

private void handleMessage(Message message) {
    try {/*from www  . j a va2 s.c  om*/
        String messageId = message.getMessageId();
        String messagePayload = unmarshalMessageBody(message.getBody());

        Map<String, String> stringifiedMessageAttributes = message.getMessageAttributes().entrySet().stream()
                .collect(Collectors.toMap(Map.Entry::getKey, e -> String.valueOf(e.getValue())));

        //SNS message attributes are stored within the SQS message body. Add them to other attributes..
        Map<String, MessageAttributeWrapper> messageAttributes = unmarshalMessageAttributes(message.getBody());
        stringifiedMessageAttributes.putAll(messageAttributes.entrySet().stream()
                .collect(Collectors.toMap(Map.Entry::getKey, e -> e.getValue().getAttributeValue())));

        MessageDescription description = MessageDescription.builder().subscriptionName(getSubscriptionName())
                .messagePayload(messagePayload).messageAttributes(stringifiedMessageAttributes)
                .pubsubSystem(pubsubSystem).ackDeadlineSeconds(60) // Set a high upper bound on message processing time.
                .retentionDeadlineSeconds(subscription.getDedupeRetentionSeconds()) // Configurable but default to 1 hour
                .build();

        AmazonMessageAcknowledger acknowledger = new AmazonMessageAcknowledger(amazonSQS, queueId, message,
                registry, getName());

        if (subscription.getMessageFormat() != AmazonPubsubProperties.MessageFormat.NONE) {
            try {
                description.setArtifacts(parseArtifacts(description.getMessagePayload(), messageId));
            } catch (FatalTemplateErrorsException e) {
                log.error("Template failed to process artifacts for message {}", message, e);
            }
        }

        if (subscription.getAlternateIdInMessageAttributes() != null
                && !subscription.getAlternateIdInMessageAttributes().isEmpty()
                && stringifiedMessageAttributes.containsKey(subscription.getAlternateIdInMessageAttributes())) {
            // Message attributes contain the unique id used for deduping
            messageId = stringifiedMessageAttributes.get(subscription.getAlternateIdInMessageAttributes());
        }

        pubsubMessageHandler.handleMessage(description, acknowledger, identity.getIdentity(), messageId);
    } catch (Exception e) {
        registry.counter(getFailedToBeHandledMetricId(e)).increment();
        log.error("Message {} from queue {} failed to be handled", message, queueId, e);
        // Todo emjburns: add dead-letter queue policy
    }
}

From source file:com.streamsets.pipeline.stage.origin.sqs.SqsConsumerWorkerCallable.java

License:Apache License

private void setSqsAttributesOnRecord(Message message, Record record, String queueUrl, String queueNamePrefix) {
    final Record.Header header = record.getHeader();

    switch (sqsAttributesOption) {
    case ALL://from w ww .  ja va2 s. c  om
        header.setAttribute(SQS_QUEUE_URL_ATTRIBUTE, queueUrl);
        Optional.of(message.getMessageAttributes()).ifPresent(attrs -> {
            attrs.forEach((name, val) -> {
                final String stringValue = val.getStringValue();
                if (stringValue != null) {
                    header.setAttribute(SQS_MESSAGE_ATTRIBUTE_PREFIX + name, stringValue);
                }
            });
        });
        final String body = message.getBody();
        if (body != null) {
            header.setAttribute(SQS_MESSAGE_BODY_ATTRIBUTE, body);
        }
        final String bodyMd5 = message.getMD5OfBody();
        if (bodyMd5 != null) {
            header.setAttribute(SQS_MESSAGE_BODY_MD5_ATTRIBUTE, bodyMd5);
        }
        final String attrsMd5 = message.getMD5OfMessageAttributes();
        if (attrsMd5 != null) {
            header.setAttribute(SQS_MESSAGE_ATTRIBUTE_MD5_ATTRIBUTE, attrsMd5);
        }
        // fall through
    case BASIC:
        header.setAttribute(SQS_MESSAGE_ID_ATTRIBUTE, message.getMessageId());
        header.setAttribute(SQS_QUEUE_NAME_PREFIX_ATTRIBUTE, queueNamePrefix);
        header.setAttribute(SQS_REGION_ATTRIBUTE, awsRegionLabel);
        break;
    case NONE:
        // empty block
        break;
    }
}

From source file:io.relution.jenkins.awssqs.SQSTrigger.java

License:Apache License

private void handleMessage(final Message message) {
    Log.info("Message received...");
    Map<String, String> jobParams = new HashMap<>();

    // add job parameters from the message (N.B. won't work post Jenkins v2+) @see https://wiki.jenkins-ci.org/display/JENKINS/Plugins+affected+by+fix+for+SECURITY-170
    for (Map.Entry<String, MessageAttributeValue> att : message.getMessageAttributes().entrySet()) {
        if (StringUtils.isNotBlank(att.getKey()) && att.getValue() != null) {
            jobParams.put("sqs_" + att.getKey(), att.getValue().getStringValue());
        }//from  www  .  j a v  a2s  . c om
    }
    jobParams.put("sqs_body", message.getBody());
    jobParams.put("sqs_messageId", message.getMessageId());
    jobParams.put("sqs_receiptHandle", message.getReceiptHandle());
    jobParams.put("sqs_bodyMD5", message.getMD5OfBody());
    startJob(jobParams);

    //        final MessageParser parser = this.messageParserFactory.createParser(message);
    //        final EventTriggerMatcher matcher = this.getEventTriggerMatcher();
    //        final List<ExecuteJenkinsJobEvent> events = parser.parseMessage(message);
    //
    //        if (matcher.matches(events, this.job)) {
    //            this.execute();
    //        }else{
    //            Log.info("Executing handleMessage when no event is matched");
    //            this.execute();
    //        }
}

From source file:org.apache.nifi.processors.aws.sqs.GetSQS.java

License:Apache License

@Override
public void onTrigger(final ProcessContext context, final ProcessSession session) {
    final String queueUrl = context.getProperty(DYNAMIC_QUEUE_URL).evaluateAttributeExpressions().getValue();

    final AmazonSQSClient client = getClient();

    final ReceiveMessageRequest request = new ReceiveMessageRequest();
    request.setAttributeNames(Collections.singleton("All"));
    request.setMessageAttributeNames(Collections.singleton("All"));
    request.setMaxNumberOfMessages(context.getProperty(BATCH_SIZE).asInteger());
    request.setVisibilityTimeout(//from  w  w  w. j  a  v  a 2 s  . c  o  m
            context.getProperty(VISIBILITY_TIMEOUT).asTimePeriod(TimeUnit.SECONDS).intValue());
    request.setQueueUrl(queueUrl);
    request.setWaitTimeSeconds(
            context.getProperty(RECEIVE_MSG_WAIT_TIME).asTimePeriod(TimeUnit.SECONDS).intValue());

    final Charset charset = Charset.forName(context.getProperty(CHARSET).getValue());

    final ReceiveMessageResult result;
    try {
        result = client.receiveMessage(request);
    } catch (final Exception e) {
        getLogger().error("Failed to receive messages from Amazon SQS due to {}", new Object[] { e });
        context.yield();
        return;
    }

    final List<Message> messages = result.getMessages();
    if (messages.isEmpty()) {
        context.yield();
        return;
    }

    final boolean autoDelete = context.getProperty(AUTO_DELETE).asBoolean();

    for (final Message message : messages) {
        FlowFile flowFile = session.create();

        final Map<String, String> attributes = new HashMap<>();
        for (final Map.Entry<String, String> entry : message.getAttributes().entrySet()) {
            attributes.put("sqs." + entry.getKey(), entry.getValue());
        }

        for (final Map.Entry<String, MessageAttributeValue> entry : message.getMessageAttributes().entrySet()) {
            attributes.put("sqs." + entry.getKey(), entry.getValue().getStringValue());
        }

        attributes.put("hash.value", message.getMD5OfBody());
        attributes.put("hash.algorithm", "md5");
        attributes.put("sqs.message.id", message.getMessageId());
        attributes.put("sqs.receipt.handle", message.getReceiptHandle());

        flowFile = session.putAllAttributes(flowFile, attributes);
        flowFile = session.write(flowFile, new OutputStreamCallback() {
            @Override
            public void process(final OutputStream out) throws IOException {
                out.write(message.getBody().getBytes(charset));
            }
        });

        session.transfer(flowFile, REL_SUCCESS);
        session.getProvenanceReporter().receive(flowFile, queueUrl);

        getLogger().info("Successfully received {} from Amazon SQS", new Object[] { flowFile });
    }

    if (autoDelete) {
        // If we want to auto-delete messages, we must fist commit the session to ensure that the data
        // is persisted in NiFi's repositories.
        session.commit();

        final DeleteMessageBatchRequest deleteRequest = new DeleteMessageBatchRequest();
        deleteRequest.setQueueUrl(queueUrl);
        final List<DeleteMessageBatchRequestEntry> deleteRequestEntries = new ArrayList<>();
        for (final Message message : messages) {
            final DeleteMessageBatchRequestEntry entry = new DeleteMessageBatchRequestEntry();
            entry.setId(message.getMessageId());
            entry.setReceiptHandle(message.getReceiptHandle());
            deleteRequestEntries.add(entry);
        }

        deleteRequest.setEntries(deleteRequestEntries);

        try {
            client.deleteMessageBatch(deleteRequest);
        } catch (final Exception e) {
            getLogger().error(
                    "Received {} messages from Amazon SQS but failed to delete the messages; these messages"
                            + " may be duplicated. Reason for deletion failure: {}",
                    new Object[] { messages.size(), e });
        }
    }

}

From source file:org.wso2.carbon.inbound.amazonsqs.AmazonSQSPollingConsumer.java

License:Open Source License

/**
 * Create connection with broker and retrieve the messages. Then inject
 * according to the registered handler./*from   www . j  av  a  2 s  .co m*/
 */
public Message poll() {
    if (logger.isDebugEnabled()) {
        logger.debug("Polling AmazonSQS messages for " + name);
    }
    try {
        if (!isConnected) {
            sqsClient = new AmazonSQSClient(this.credentials);
            isConnected = true;
        }
        if (sqsClient == null) {
            logger.error("AmazonSQS Inbound endpoint " + name + " unable to get a connection.");
            isConnected = false;
            return null;
        }
        List<Message> messages;
        receiveMessageRequest.setMessageAttributeNames(attributeNames);
        messages = sqsClient.receiveMessage(receiveMessageRequest).getMessages();
        if (!messages.isEmpty()) {
            for (Message message : messages) {
                boolean commitOrRollbacked;
                if (logger.isDebugEnabled()) {
                    logger.debug(
                            "Injecting AmazonSQS message to the sequence : " + injectingSeq + " of " + name);
                }
                //Get the content type of the message.
                if (message.getMessageAttributes().containsKey(AmazonSQSConstants.CONTENT_TYPE)) {
                    contentType = message.getMessageAttributes().get(AmazonSQSConstants.CONTENT_TYPE)
                            .getStringValue();
                    if (contentType.trim().equals("") || contentType.equals("null")) {
                        contentType = AmazonSQSConstants.DEFAULT_CONTENT_TYPE;
                    }
                } else {
                    contentType = properties.getProperty(AmazonSQSConstants.CONTENT_TYPE);
                }
                if (logger.isDebugEnabled()) {
                    logger.debug("Loading the Content-type : " + contentType + " for " + name);
                }
                commitOrRollbacked = injectMessage(message.getBody(), contentType);
                if (commitOrRollbacked) {
                    messageReceiptHandle = message.getReceiptHandle();
                    sqsClient.deleteMessage(new DeleteMessageRequest(destination, messageReceiptHandle));
                }
            }
        } else {
            return null;
        }
    } catch (AmazonServiceException e) {
        throw new SynapseException("Caught an AmazonServiceException, which means your "
                + "request made it to Amazon SQS, but was rejected with an" + "error response for some reason.",
                e);
    } catch (AmazonClientException e) {
        throw new SynapseException("Caught an AmazonClientException, which means the client"
                + " encountered a serious internal problem while trying to communicate with SQS, "
                + "such as not being able to access the network.", e);
    }
    return null;
}

From source file:smartthings.brave.sqs.TracingAmazonSQSClient.java

License:Apache License

@Override
public ReceiveMessageResult receiveMessage(ReceiveMessageRequest receiveMessageRequest) {
    receiveMessageRequest = receiveMessageRequest.withMessageAttributeNames(tracing.propagation().keys());

    ReceiveMessageResult result = delegate.receiveMessage(receiveMessageRequest);

    // complete in flight one-way spans for all received messages
    for (Message message : result.getMessages()) {
        TraceContextOrSamplingFlags traceContextOrSamplingFlags = extractor
                .extract(message.getMessageAttributes());
        TraceContext ctx = traceContextOrSamplingFlags.context();
        Span oneWay = withEndpoint((ctx != null) ? tracing.tracer().joinSpan(ctx)
                : tracing.tracer().newTrace(traceContextOrSamplingFlags.samplingFlags()));

        oneWay.kind(Span.Kind.SERVER);//from   w w  w . ja  v a  2  s  .c  om
        parser.response(result, oneWay);
        oneWay.annotate("receive-" + parser.spanName(receiveMessageRequest.getQueueUrl()));
        oneWay.start().flush();
    }

    return result;
}