List of usage examples for com.amazonaws.services.s3.model ObjectListing getObjectSummaries
public List<S3ObjectSummary> getObjectSummaries()
From source file:S3ClientSideEncryptionWithSymmetricMasterKey.java
License:Apache License
private static void deleteBucketAndAllContents(AmazonS3 client) { System.out.println("Deleting S3 bucket: " + bucketName); ObjectListing objectListing = client.listObjects(bucketName); while (true) { for (Iterator<?> iterator = objectListing.getObjectSummaries().iterator(); iterator.hasNext();) { S3ObjectSummary objectSummary = (S3ObjectSummary) iterator.next(); client.deleteObject(bucketName, objectSummary.getKey()); }/*from www. j a v a 2 s.com*/ if (objectListing.isTruncated()) { objectListing = client.listNextBatchOfObjects(objectListing); } else { break; } } ; client.deleteBucket(bucketName); }
From source file:BackupManager.java
License:Open Source License
public static void main(String[] args) throws IOException { AmazonS3 s3 = new AmazonS3Client( new PropertiesCredentials(BackupManager.class.getResourceAsStream("AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; System.out.println("==========================================="); System.out.println("Getting Started with Amazon S3"); System.out.println("===========================================\n"); try {/*www . j av a 2 s.c o m*/ /* * Create a new S3 bucket - Amazon S3 bucket names are globally unique, * so once a bucket name has been taken by any user, you can't create * another bucket with that same name. * * You can optionally specify a location for your bucket if you want to * keep your data closer to your applications or users. */ System.out.println("Creating bucket " + bucketName + "\n"); s3.createBucket(bucketName); /* * List the buckets in your account */ System.out.println("Listing buckets"); for (Bucket bucket : s3.listBuckets()) { System.out.println(" - " + bucket.getName()); } System.out.println(); /* * Upload an object to your bucket - You can easily upload a file to * S3, or upload directly an InputStream if you know the length of * the data in the stream. You can also specify your own metadata * when uploading to S3, which allows you set a variety of options * like content-type and content-encoding, plus additional metadata * specific to your applications. */ System.out.println("Uploading a new object to S3 from a file\n"); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); /* * Download an object - When you download an object, you get all of * the object's metadata and a stream from which to read the contents. * It's important to read the contents of the stream as quickly as * possibly since the data is streamed directly from Amazon S3 and your * network connection will remain open until you read all the data or * close the input stream. * * GetObjectRequest also supports several other options, including * conditional downloading of objects based on modification times, * ETags, and selectively downloading a range of an object. */ System.out.println("Downloading an object"); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); displayTextInputStream(object.getObjectContent()); /* * List objects in your bucket by prefix - There are many options for * listing the objects in your bucket. Keep in mind that buckets with * many objects might truncate their results when listing their objects, * so be sure to check if the returned object listing is truncated, and * use the AmazonS3.listNextBatchOfObjects(...) operation to retrieve * additional results. */ System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); /* * Delete an object - Unless versioning has been turned on for your bucket, * there is no way to undelete an object, so use caution when deleting objects. */ System.out.println("Deleting an object\n"); s3.deleteObject(bucketName, key); /* * Delete a bucket - A bucket must be completely empty before it can be * deleted, so remember to delete any objects from your buckets before * you try to delete them. */ System.out.println("Deleting bucket " + bucketName + "\n"); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:AwsConsoleApp.java
License:Open Source License
public static void main(String[] args) throws Exception { System.out.println("==========================================="); System.out.println("Welcome to the AWS Java SDK!"); System.out.println("==========================================="); init();/* w w w . j a va 2 s . co m*/ /* * Amazon EC2 * * The AWS EC2 client allows you to create, delete, and administer * instances programmatically. * * In this sample, we use an EC2 client to get a list of all the * availability zones, and all instances sorted by reservation id. */ try { DescribeAvailabilityZonesResult availabilityZonesResult = ec2.describeAvailabilityZones(); System.out.println("You have access to " + availabilityZonesResult.getAvailabilityZones().size() + " Availability Zones."); DescribeInstancesResult describeInstancesRequest = ec2.describeInstances(); List<Reservation> reservations = describeInstancesRequest.getReservations(); Set<Instance> instances = new HashSet<Instance>(); for (Reservation reservation : reservations) { instances.addAll(reservation.getInstances()); } System.out.println("You have " + instances.size() + " Amazon EC2 instance(s) running."); } catch (AmazonServiceException ase) { System.out.println("Caught Exception: " + ase.getMessage()); System.out.println("Reponse Status Code: " + ase.getStatusCode()); System.out.println("Error Code: " + ase.getErrorCode()); System.out.println("Request ID: " + ase.getRequestId()); } /* * Amazon SimpleDB * * The AWS SimpleDB client allows you to query and manage your data * stored in SimpleDB domains (similar to tables in a relational DB). * * In this sample, we use a SimpleDB client to iterate over all the * domains owned by the current user, and add up the number of items * (similar to rows of data in a relational DB) in each domain. */ try { ListDomainsRequest sdbRequest = new ListDomainsRequest().withMaxNumberOfDomains(100); ListDomainsResult sdbResult = sdb.listDomains(sdbRequest); int totalItems = 0; for (String domainName : sdbResult.getDomainNames()) { DomainMetadataRequest metadataRequest = new DomainMetadataRequest().withDomainName(domainName); DomainMetadataResult domainMetadata = sdb.domainMetadata(metadataRequest); totalItems += domainMetadata.getItemCount(); } System.out.println("You have " + sdbResult.getDomainNames().size() + " Amazon SimpleDB domain(s)" + "containing a total of " + totalItems + " items."); } catch (AmazonServiceException ase) { System.out.println("Caught Exception: " + ase.getMessage()); System.out.println("Reponse Status Code: " + ase.getStatusCode()); System.out.println("Error Code: " + ase.getErrorCode()); System.out.println("Request ID: " + ase.getRequestId()); } /* * Amazon S3 * * The AWS S3 client allows you to manage buckets and programmatically * put and get objects to those buckets. * * In this sample, we use an S3 client to iterate over all the buckets * owned by the current user, and all the object metadata in each * bucket, to obtain a total object and space usage count. This is done * without ever actually downloading a single object -- the requests * work with object metadata only. */ try { List<Bucket> buckets = s3.listBuckets(); long totalSize = 0; int totalItems = 0; for (Bucket bucket : buckets) { /* * In order to save bandwidth, an S3 object listing does not * contain every object in the bucket; after a certain point the * S3ObjectListing is truncated, and further pages must be * obtained with the AmazonS3Client.listNextBatchOfObjects() * method. */ ObjectListing objects = s3.listObjects(bucket.getName()); do { for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) { totalSize += objectSummary.getSize(); totalItems++; } objects = s3.listNextBatchOfObjects(objects); } while (objects.isTruncated()); } System.out.println("You have " + buckets.size() + " Amazon S3 bucket(s), " + "containing " + totalItems + " objects with a total size of " + totalSize + " bytes."); } catch (AmazonServiceException ase) { /* * AmazonServiceExceptions represent an error response from an AWS * services, i.e. your request made it to AWS, but the AWS service * either found it invalid or encountered an error trying to execute * it. */ System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { /* * AmazonClientExceptions represent an error that occurred inside * the client on the local host, either while trying to send the * request to AWS or interpret the response. For example, if no * network connection is available, the client won't be able to * connect to AWS to execute a request and will throw an * AmazonClientException. */ System.out.println("Error Message: " + ace.getMessage()); } }
From source file:S3Sample.java
License:Open Source License
public static void main(String[] args) throws IOException { /*//from w w w . j av a 2 s . com * Important: Be sure to fill in your AWS access credentials in the * AwsCredentials.properties file before you try to run this * sample. * http://aws.amazon.com/security-credentials */ AmazonS3 s3 = new AmazonS3Client( new PropertiesCredentials(S3Sample.class.getResourceAsStream("AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; System.out.println("==========================================="); System.out.println("Getting Started with Amazon S3"); System.out.println("===========================================\n"); try { /* * Create a new S3 bucket - Amazon S3 bucket names are globally unique, * so once a bucket name has been taken by any user, you can't create * another bucket with that same name. * * You can optionally specify a location for your bucket if you want to * keep your data closer to your applications or users. */ System.out.println("Creating bucket " + bucketName + "\n"); s3.createBucket(bucketName); /* * List the buckets in your account */ System.out.println("Listing buckets"); for (Bucket bucket : s3.listBuckets()) { System.out.println(" - " + bucket.getName()); } System.out.println(); /* * Upload an object to your bucket - You can easily upload a file to * S3, or upload directly an InputStream if you know the length of * the data in the stream. You can also specify your own metadata * when uploading to S3, which allows you set a variety of options * like content-type and content-encoding, plus additional metadata * specific to your applications. */ System.out.println("Uploading a new object to S3 from a file\n"); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); /* * Download an object - When you download an object, you get all of * the object's metadata and a stream from which to read the contents. * It's important to read the contents of the stream as quickly as * possibly since the data is streamed directly from Amazon S3 and your * network connection will remain open until you read all the data or * close the input stream. * * GetObjectRequest also supports several other options, including * conditional downloading of objects based on modification times, * ETags, and selectively downloading a range of an object. */ System.out.println("Downloading an object"); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); displayTextInputStream(object.getObjectContent()); /* * List objects in your bucket by prefix - There are many options for * listing the objects in your bucket. Keep in mind that buckets with * many objects might truncate their results when listing their objects, * so be sure to check if the returned object listing is truncated, and * use the AmazonS3.listNextBatchOfObjects(...) operation to retrieve * additional results. */ System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); /* * Delete an object - Unless versioning has been turned on for your bucket, * there is no way to undelete an object, so use caution when deleting objects. */ System.out.println("Deleting an object\n"); s3.deleteObject(bucketName, key); /* * Delete a bucket - A bucket must be completely empty before it can be * deleted, so remember to delete any objects from your buckets before * you try to delete them. */ System.out.println("Deleting bucket " + bucketName + "\n"); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:S3ClientSideEncryptionAsymmetricMasterKey.java
License:Apache License
private static void deleteBucketAndAllContents(AmazonS3 client) { System.out.println("Deleting S3 bucket: " + bucketName); ObjectListing objectListing = client.listObjects(bucketName); while (true) { for (Iterator<?> iterator = objectListing.getObjectSummaries().iterator(); iterator.hasNext();) { S3ObjectSummary objectSummary = (S3ObjectSummary) iterator.next(); client.deleteObject(bucketName, objectSummary.getKey()); }//from ww w. j a va 2s .c o m if (objectListing.isTruncated()) { objectListing = client.listNextBatchOfObjects(objectListing); } else { break; } } client.deleteBucket(bucketName); }
From source file:AmazonS3Handler.java
License:Open Source License
public void listObject() { try {//from ww w. jav a 2 s.c o m // System.out.println("Creating bucket " + bucketName + "\n"); // s3.createBucket(bucketName); // System.out.println("Listing buckets"); // for (Bucket bucket : s3.listBuckets()) { // System.out.println(" - " + bucket.getName()); // } // System.out.println(); // System.out.println("Uploading a new object to S3 from a file\n"); // s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile()).withCannedAcl(CannedAccessControlList.PublicRead)); // System.out.println("Downloading an object"); // S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); // System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); // displayTextInputStream(object.getObjectContent()); System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); // System.out.println("Deleting an object\n"); // s3.deleteObject(bucketName, key); // System.out.println("Deleting bucket " + bucketName + "\n"); // s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:alluxio.underfs.s3a.S3AUnderFileSystem.java
License:Apache License
/** * Gets the metadata associated with a non-root key if it represents a folder. This method will * return null if the key is not a folder. If the key exists as a prefix but does not have the * folder dummy file, a folder dummy file will be created. * * @param key the key to get the metadata for * @return the metadata of the folder, or null if the key does not represent a folder *//*www . j av a2s.co m*/ private ObjectMetadata getFolderMetadata(String key) { Preconditions.checkArgument(!isRoot(key)); String keyAsFolder = convertToFolderName(stripPrefixIfPresent(key)); ObjectMetadata meta = null; try { meta = mClient.getObjectMetadata(mBucketName, keyAsFolder); // If no exception is thrown, the key exists as a folder } catch (AmazonClientException e) { // It is possible that the folder has not been encoded as a _$folder$ file try { String dir = stripPrefixIfPresent(key); String dirPrefix = PathUtils.normalizePath(dir, PATH_SEPARATOR); // Check if anything begins with <folder_path>/ ObjectListing objs = mClient.listObjects(mBucketName, dirPrefix); // If there are, this is a folder and we can create the necessary metadata if (objs.getObjectSummaries().size() > 0) { mkdirsInternal(dir); meta = mClient.getObjectMetadata(mBucketName, keyAsFolder); } } catch (AmazonClientException ace) { return null; } } return meta; }
From source file:aws.example.s3.DeleteBucket.java
License:Open Source License
public static void main(String[] args) { final String USAGE = "\n" + "To run this example, supply the name of an S3 bucket\n" + "\n" + "Ex: DeleteBucket <bucketname>\n"; if (args.length < 1) { System.out.println(USAGE); System.exit(1);/* ww w . ja v a 2 s .c o m*/ } String bucket_name = args[0]; System.out.println("Deleting S3 bucket: " + bucket_name); final AmazonS3 s3 = new AmazonS3Client(); try { System.out.println(" - removing objects from bucket"); ObjectListing object_listing = s3.listObjects(bucket_name); while (true) { for (Iterator<?> iterator = object_listing.getObjectSummaries().iterator(); iterator.hasNext();) { S3ObjectSummary summary = (S3ObjectSummary) iterator.next(); s3.deleteObject(bucket_name, summary.getKey()); } // more object_listing to retrieve? if (object_listing.isTruncated()) { object_listing = s3.listNextBatchOfObjects(object_listing); } else { break; } } ; System.out.println(" - removing versions from bucket"); VersionListing version_listing = s3.listVersions(new ListVersionsRequest().withBucketName(bucket_name)); while (true) { for (Iterator<?> iterator = version_listing.getVersionSummaries().iterator(); iterator.hasNext();) { S3VersionSummary vs = (S3VersionSummary) iterator.next(); s3.deleteVersion(bucket_name, vs.getKey(), vs.getVersionId()); } if (version_listing.isTruncated()) { version_listing = s3.listNextBatchOfVersions(version_listing); } else { break; } } System.out.println(" OK, bucket ready to delete!"); s3.deleteBucket(bucket_name); } catch (AmazonServiceException e) { System.err.println(e.getErrorMessage()); System.exit(1); } System.out.println("Done!"); }
From source file:aws.example.s3.ListObjects.java
License:Open Source License
public static void main(String[] args) { final String USAGE = "\n" + "To run this example, supply the name of a bucket to list!\n" + "\n" + "Ex: ListObjects <bucket-name>\n"; if (args.length < 1) { System.out.println(USAGE); System.exit(1);//from www . j a v a2s.c o m } String bucket_name = args[0]; System.out.format("Objects in S3 bucket %s:\n", bucket_name); final AmazonS3 s3 = new AmazonS3Client(); ObjectListing ol = s3.listObjects(bucket_name); List<S3ObjectSummary> objects = ol.getObjectSummaries(); for (S3ObjectSummary os : objects) { System.out.println("* " + os.getKey()); } }
From source file:aws.sample.AwsConsoleApp.java
License:Open Source License
public static void main(String[] args) throws Exception { System.out.println("==========================================="); System.out.println("Welcome to the AWS Java SDK!"); System.out.println("==========================================="); init();/* w w w . j av a 2s.c om*/ /* * Amazon EC2 * * The AWS EC2 client allows you to create, delete, and administer instances programmatically. * * In this sample, we use an EC2 client to get a list of all the availability zones, and all instances sorted by reservation id. */ try { DescribeAvailabilityZonesResult availabilityZonesResult = ec2.describeAvailabilityZones(); System.out.println("You have access to " + availabilityZonesResult.getAvailabilityZones().size() + " Availability Zones."); DescribeInstancesResult describeInstancesRequest = ec2.describeInstances(); List<Reservation> reservations = describeInstancesRequest.getReservations(); Set<Instance> instances = new HashSet<Instance>(); for (Reservation reservation : reservations) { instances.addAll(reservation.getInstances()); } System.out.println("You have " + instances.size() + " Amazon EC2 instance(s) running."); } catch (AmazonServiceException ase) { System.out.println("Caught Exception: " + ase.getMessage()); System.out.println("Reponse Status Code: " + ase.getStatusCode()); System.out.println("Error Code: " + ase.getErrorCode()); System.out.println("Request ID: " + ase.getRequestId()); } /* * Amazon SimpleDB * * The AWS SimpleDB client allows you to query and manage your data stored in SimpleDB domains (similar to tables in a relational DB). * * In this sample, we use a SimpleDB client to iterate over all the domains owned by the current user, and add up the number of items (similar to rows of data in a relational DB) in each domain. */ try { ListDomainsRequest sdbRequest = new ListDomainsRequest().withMaxNumberOfDomains(100); ListDomainsResult sdbResult = sdb.listDomains(sdbRequest); int totalItems = 0; for (String domainName : sdbResult.getDomainNames()) { DomainMetadataRequest metadataRequest = new DomainMetadataRequest().withDomainName(domainName); DomainMetadataResult domainMetadata = sdb.domainMetadata(metadataRequest); totalItems += domainMetadata.getItemCount(); } System.out.println("You have " + sdbResult.getDomainNames().size() + " Amazon SimpleDB domain(s)" + "containing a total of " + totalItems + " items."); } catch (AmazonServiceException ase) { System.out.println("Caught Exception: " + ase.getMessage()); System.out.println("Reponse Status Code: " + ase.getStatusCode()); System.out.println("Error Code: " + ase.getErrorCode()); System.out.println("Request ID: " + ase.getRequestId()); } /* * Amazon S3 * * The AWS S3 client allows you to manage buckets and programmatically put and get objects to those buckets. * * In this sample, we use an S3 client to iterate over all the buckets owned by the current user, and all the object metadata in each bucket, to obtain a total object and space usage count. This is done without ever actually downloading a single object -- the requests work with object metadata only. */ try { List<Bucket> buckets = s3.listBuckets(); long totalSize = 0; int totalItems = 0; for (Bucket bucket : buckets) { /* * In order to save bandwidth, an S3 object listing does not contain every object in the bucket; after a certain point the S3ObjectListing is truncated, and further pages must be obtained with the AmazonS3Client.listNextBatchOfObjects() method. */ ObjectListing objects = s3.listObjects(bucket.getName()); do { for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) { totalSize += objectSummary.getSize(); totalItems++; } objects = s3.listNextBatchOfObjects(objects); } while (objects.isTruncated()); } System.out.println("You have " + buckets.size() + " Amazon S3 bucket(s), " + "containing " + totalItems + " objects with a total size of " + totalSize + " bytes."); } catch (AmazonServiceException ase) { /* * AmazonServiceExceptions represent an error response from an AWS services, i.e. your request made it to AWS, but the AWS service either found it invalid or encountered an error trying to execute it. */ System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { /* * AmazonClientExceptions represent an error that occurred inside the client on the local host, either while trying to send the request to AWS or interpret the response. For example, if no network connection is available, the client won't be able to connect to AWS to execute a request and will throw an AmazonClientException. */ System.out.println("Error Message: " + ace.getMessage()); } }