Managing S3-Compatible Storage Using CLI Tool

Most S3-compatible storage providers like UpCloud and DigitalOcean provide a dashboard for managing our storage. But, usually, we face some browser or web-related issues in certain conditions for example when we try to upload large amounts of files. There are some CLI tools out there that we can use for managing our storage like uploading files, migrating files to another bucket, etc.

One of the popular CLI tools is S3cmd. For instance, I use an object storage service provided by UpCloud. For this past year, I migrated many of my services from AWS and DigitalOcean to UpCloud because of its cost and performance. I found that UpCloud actively develops new features or services and improves its infrastructure performance.

To install S3cmd, we need to have Python and PIP in our machine. After that, we can run the following command to install S3cmd.

pip install s3cmd

Then, we can configure the tool by running the following command. Four fields are important in our case: access key ID, secret access key, storage endpoint address, and address templating. In UpCloud, the endpoint address is like your-storage-name.sg-sin1.upcloudobjects.com. While the address templating can be like %(bucket).your-storage-name.sg-sin1.upcloudobjects.com. It will let us access any bucket just by using this syntax: s3://your-bucket-name, without the complete address.

s3cmd --configure

After finishing the configuration, we can start to utilize the tool. The following example shows how to upload several files stored in a local folder to a bucket.

s3cmd put --recursive /path/to/source-folder/ s3://your-bucket-name/target-folder/

We should remember the trailing slash on the source folder path. It will copy only the contents of the source folder to the target folder. If there is no trailing slash, it will copy the folder and its content into the target folder.

We can also move files from one folder to another folder in our bucket.

s3cmd mv --recursive s3://your-bucket-name/source-folder/ s3://your-bucket-name/target-folder/

The following command is used to download files from the bucket.

s3cmd get s3://your-bucket-name/images/source.jpg /path/to/images/target.jpg

We can check the status of disk usage (e.g. size and object number) of a folder in our bucket by the following command.

s3cmd du s3://your-bucket-name/folder

There are many more commands that we can explore in the manuals.

Comments

Deploying a Web Server on UpCloud using Terraform Modules

In my earlier post , I shared an example of deploying UpCloud infrastructure using Terraform from scratch. In this post, I want to share how to deploy the infrastructure using available Terraform modules to speed up the set-up process, especially for common use cases like preparing a web server. For instance, our need is to deploy a website with some conditions as follows. The website can be accessed through HTTPS. If the request is HTTP, it will be redirected to HTTPS. There are 2 domains, web1.yourdomain.com and web2.yourdomain.com . But, users should be redirected to "web2" if they are visiting "web1". There are 4 main modules that we need to set up the environment. Private network. It allows the load balancer to connect with the server and pass the traffic. Server. It is used to host the website. Load balancer. It includes backend and frontend configuration. Dynamic certificate. It is requ...

Rangkaian Sensor Infrared dengan Photo Dioda

Keunggulan photodioda dibandingkan LDR adalah photodioda lebih tidak rentan terhadap noise karena hanya menerima sinar infrared, sedangkan LDR menerima seluruh cahaya yang ada termasuk infrared. Rangkaian yang akan kita gunakan adalah seperti gambar di bawah ini. Pada saat intensitas Infrared yang diterima Photodiode besar maka tahanan Photodiode menjadi kecil, sedangkan jika intensitas Infrared yang diterima Photodiode kecil maka tahanan yang dimiliki photodiode besar. Jika tahanan photodiode kecil maka tegangan V- akan kecil . Misal tahanan photodiode mengecil menjadi 10kOhm. Maka dengan teorema pembagi tegangan: V- = Rrx/(Rrx + R2) x Vcc V- = 10 / (10+10) x Vcc V- = (1/2) x 5 Volt V- = 2.5 Volt Sedangkan jika tahanan photodiode besar maka tegangan V- akan besar (mendekati nilai Vcc). Misal tahanan photodiode menjadi 150kOhm. Maka dengan teorema pembagi tegangan: V- = Rrx/(Rrx + R2) x Vcc V- = 150 / (1...

What's Good About Strapi, a Headless CMS

Recently, I've been revisiting Strapi as a solution for building backend systems. I still think this headless CMS can be quite useful in certain cases, especially for faster prototyping or creating common websites like company profiles or e-commerce platforms . It might even have the potential to handle more complex systems. With the release of version 5, I'm curious to know what updates it brings. Strapi has launched a new documentation page, and it already feels like an improvement in navigation and content structure compared to the previous version. That said, there's still room for improvement, particularly when it comes to use cases and best practices for working with Strapi. In my opinion, Strapi stands out with some compelling features that could catch developers' attention. I believe three key aspects of Strapi offer notable advantages. First, the content-type builder feature lets us design the data structure of an entity or database model , including ...

How To Verify Phone Number for Free Using WhatsApp

If you have a product or business that maintains user information like phone numbers, verifying the validity or ownership of the phone number could become important, as the phone number can be used as an authentication method or targeted marketing channel. The typical phone verification procedure is by generating a code or OTP in our application, sending that OTP to the user's phone, and then the user should insert the OTP in our application for verification. The OTP can be sent to the users through services like SMS or WhatsApp that require a valid phone number. For internet-based communication, WhatsApp has become the de facto standard for sending the OTP. WhatsApp requires its users to have a valid phone number during account creation, and it already has a huge number of users, approximately 3 billion in 2025. Using that common procedure, WhatsApp will charge us for each OTP sent. The cost depends on the country of the target phone number. For Indonesia...

Armin or Commander Erwin

In the moment of conflict in the scout team, who will be revived?

Free Cloud Services from UpCloud

Although I typically deploy my development environment or experimental services on UpCloud , I do not always stay updated on its announcements. Recently, I discovered that UpCloud has introduced a new plan called the Essentials plan, which enables certain cloud services to be deployed at no cost. The complimentary services are generally associated with network components or serve as the foundation for other cloud services. This feature is particularly useful when retaining foundational services, such as a load balancer, is necessary, while tearing down all services and reconfiguring the DNS and other application settings each time we temporarily clean up infrastructure to reduce costs is undesirable. When reviewing the service specifications of the cloud services in the Essentials plan, they appear to be very similar to those in the Development plan. The difference in service levels is unclear, but it could be related to hardware or resource allocation. For instance, the loa...

LUKI NOTES

Search This Blog