Forem: Chetan

Cloudinary to s3 bucket db changes

Chetan — Tue, 27 Feb 2024 06:02:15 +0000

If you have a database collection that has url field , containing cloudinary or any cloud based platform links and you want to shift/ migrate the urls specifically to s3 bucket then you can use this script.

     query = { $regex: /^https:\/\/res\.cloudinary\.com/i }
     const urls = await Audio.find(query).toArray(); 

        const audioList = [];

        // Iterate over each document and process one at a time
        for (const doc of urls) {
            const _id = doc._id;
            const url = doc.url;

                  const filename = path.join("L:/Check Audios", `${_id}.mp3`);

//Function to download and save audio locally 
            await downloadAndSaveAudio(url, filePath);

            // Upload the audio file to S3
            const uploadParams = {
                Bucket: bucketName,
                Key: filename,
                Body: fs.createReadStream(filename),
                ContentType: 'audio/mp3'
            };
            const uploadResponse = await s3Client.send(new PutObjectCommand(uploadParams));

            // Generate S3 URL
            const s3Url = `https://${bucketName}.s3.${bucketRegion}.amazonaws.com/${filename}`;

            // Update the database with the S3 URL
            await Audio.updateOne({ _id: _id }, { $set: { url: s3Url } });

            // Push information to audioList
            audioList.push({ _id: _id, s3Url: s3Url });
        }

Cloudinary to s3 bucket db changes

Chetan — Tue, 27 Feb 2024 06:02:14 +0000

     query = { $regex: /^https:\/\/res\.cloudinary\.com/i }
     const urls = await Audio.find(query).toArray(); 

        const audioList = [];

        // Iterate over each document and process one at a time
        for (const doc of urls) {
            const _id = doc._id;
            const url = doc.url;

                  const filename = path.join("L:/Check Audios", `${_id}.mp3`);

//Function to download and save audio locally 
            await downloadAndSaveAudio(url, filePath);

            // Upload the audio file to S3
            const uploadParams = {
                Bucket: bucketName,
                Key: filename,
                Body: fs.createReadStream(filename),
                ContentType: 'audio/mp3'
            };
            const uploadResponse = await s3Client.send(new PutObjectCommand(uploadParams));

            // Generate S3 URL
            const s3Url = `https://${bucketName}.s3.${bucketRegion}.amazonaws.com/${filename}`;

            // Update the database with the S3 URL
            await Audio.updateOne({ _id: _id }, { $set: { url: s3Url } });

            // Push information to audioList
            audioList.push({ _id: _id, s3Url: s3Url });
        }