Skip to content

Interested in AI, automation, blockchain, web and apps

Seoul, KR--:-- GMT
Let’s Talk

Work/Automation/KO

Automated Product Listing Tool for Open Markets

Automated Product Listing Tool for Open Markets

Sending storefront products straight into the Coupang seller admin

Twelve option rows, typed by hand

The same products had to be listed in two places: a brand mall and a marketplace. The products were already live on the brand mall, so the work was re-entering each one into the marketplace seller admin. Three colors and four sizes means twelve option rows, and every listing also needs the representative and detail images, nine apparel disclosure fields, return center and outbound shipping codes, and the delivery fee policy. Orders were only visible after logging into the seller admin. The scope was two things: register a product on the marketplace from its brand mall URL, and send a mobile notification when a new order comes in.

System boundary and external dependencies
System boundary and external dependencies

A signed request, not a click

I dropped the Selenium approach that drove the admin screen and moved to the Open API with HMAC signing

The first version clicked through the admin form with absolute XPaths in main.py. It worked, but one layout change broke /html/body/div[1]/div[2]/... wholesale, and category selection meant a human memorizing li indexes. POSTing one payload breaks far less often. The Open API has no equivalent of the admin's "apply to all rows" button, so I had to build the items array myself with every field filled in per option.

I split scraping out of the React app into an Express + Puppeteer server (nodeapp)

A browser can't launch headless Chrome, and the brand mall doesn't open CORS either. It came down to needing one small server running locally. There's a leftover in package.json where the browser field maps fs and child_process to false, which is the scar from trying to bundle Puppeteer into the front end.

I assembled the marketplace payload from base / itemBase constants using immer's produce

Most of the 60 fields are fixed per seller account. Freezing a template and overwriting only what changes through a draft was the safer shape. Account-bound values like returnCenterCode and outboundShippingPlaceCode had no home other than hardcoded constants.

Order alerts run on a 10-second poll and a pickle snapshot diff, not a webhook

The Open API doesn't offer an order webhook. So the bot pulls every ACCEPT order from the last 28 days, compares against the previous list, and sends only what's new. There was no server, so it had to run permanently on the operator's laptop, and completing the list means recursing through the whole nextToken pagination.

Deployment and infrastructure
Deployment and infrastructure

Four pieces on one laptop

Four pieces do the work. The React app (coupang-helper) draws the screen, and nodeapp handles two jobs. POST /scrape opens the brand mall product page with Puppeteer and returns the product name, prices, images, disclosure table, and search keywords as JSON; POST / forwards the payload the app built straight on to api-gateway.coupang.com.

The moment the scrape result lands in state, the app builds the color × size combinations inside a useEffect, fills items, and finishes the body with the chosen category code and product code. After registration it clears the screen with window.location.reload() so the next product can go in. That's the registration line. The other two handle what happens after. coupang-edit is a bulk edit script that takes a list of product ids, GETs each full product document, changes one field, and PUTs it back. coupang_bot hits the order lookup API every 10 seconds and pushes anything that differs from the orders.txt snapshot to Telegram. The fifth piece, coupang-product, is a prototype that redraws the same form in Material-UI, and it stops there with no submit handler.

Core data model
Core data model

Sign in the browser, let the proxy pass the headers through untouched

The first picture that comes to mind for wiring up a marketplace API is this: keep the secret on the server, have the server take the body, build the signature, and make the call on your behalf. But that path requires both sides to guarantee that the schema the screen sends and the signing path the server reconstructs always agree. I was editing the payload several times a day at that point. And calling the gateway directly from the browser runs into CORS.

I built the signature in the browser. The key point is that what gets signed is the path of the final destination, not the address the request is actually sent to. I build the header with getAuthHeader('POST', 'https://api-gateway.coupang.com/.../seller-products', body) and then send the request itself to http://localhost:4000. On the Express side, app.post('/') leaves the body alone and re-sends it carrying only authorization and x-requested-by as they came.

The CEA HmacSHA256 signature hashes only timestamp + method + path + queryString. Neither the host nor the body goes into it. So swapping the host in the middle leaves the signature valid. That meant the proxy never had to know the key and the signing logic could live in exactly one place, hmac.js. The flip side is that a signature that excludes the body can't catch body tampering. That was an acceptable trade for a tool that only ever ran inside one laptop.

Don't download the image, change one segment of its URL

The Selenium version I wrote first was that naive method in the flesh. It downloaded each detail image to the laptop with urllib.request.urlretrieve, pushed the file path into the admin dropzone's input with send_keys, and waited for the uploaded li to appear. Five images meant five round trips, and if the spinner was slow to disappear the next click failed outright.

The API version never touches a single byte of image data. I split the thumbnail URL scraped from the gallery on /, replace the eighth segment with 600, and pass that string as vendorPath. The first image gets REPRESENTATION and the rest get DETAIL, with imageOrder attached.

The brand mall's image server encoded the size inside the path. Change one segment and you have the 600px address of the same image. On top of that, the marketplace downloads and stores whatever address it gets in vendorPath on its own side, so my laptop only has to assemble a string. The upload round trips and the spinner waits disappeared entirely, and the failure points went with them.

What a user can do

Log into the seller admin and get to the product registration screen
Log into the seller admin and get to the product registration screen

Paste a brand mall product URL and pull in the product data
Paste a brand mall product URL and pull in the product data

Pick the display category and set the product code
Pick the display category and set the product code

Sign the assembled payload and register the product
Sign the assembled payload and register the product

Fill the admin form directly with the old version to register
Fill the admin form directly with the old version to register

Sweep the already registered products and fix them in one pass
Sweep the already registered products and fix them in one pass

Get new orders on Telegram as they come in
Get new orders on Telegram as they come in

Send slash now to get the current paid-order list as a file
Send slash now to get the current paid-order list as a file

Look over the category and URL form on the prototype screen
Look over the category and URL form on the prototype screen

1 / 1

API keys still in the source

The biggest problem is the keys. src/helper/hmac.js and config.py carry the access key, the secret, and the Telegram token in plain sight, and the Selenium version has the admin password in code too. Excluding config.py in .gitignore was the whole of my protection. If I built this again I'd lay down environment variables first and move signing to the server.

Second is the code that swallows failures. The proxy returns { r: true } even when the gateway answers 4xx, and the screen just reloads. The operator had no way to know whether the listing went through, so they ended up opening the admin to check anyway. Simply showing the response code and the sellerProductId on screen would have changed that. I didn't handle everything gracefully, but at minimum, showing the result to a person would have been worth a lot.

Last, the scraping is bound entirely to DOM selectors. It was a tool that would stop the day the brand mall got redesigned, and that is exactly how its life ended. The selectors should have lived in config outside the code, with the screen showing which values came back empty.

Read next

Bank, Card and Investment Aggregation App

Broccoli — 2019