How to Deduplicate Instagram Follower Lists Without Losing Provenance

When combining Instagram Followers or Following files, keep the originals and create an ID-first key in a working copy. Use id when it is present. Only when ID is missing should you fall back to a trimmed, lowercase username without a leading @.

Then use COUNTIF to flag first and repeated appearances, review conflicts, and remove duplicates only after you have preserved source provenance.

Web Tidy already deduplicates a single task by id by default. This workflow is mainly for combining exports from multiple dates, source profiles, or tools.

Define what “duplicate” means#

This page handles data cleaning only. It does not calculate non-followbacks, unfollowers, or common followers. Those are separate comparison tasks after cleaning.

Why full name and profile URL are weak keys#

• fullName is not unique.
• profileUrl usually contains the username and changes when that username changes.
• username is readable but mutable, so it should be a fallback.
• id is the preferred match key, but long numeric IDs must be imported as text before a spreadsheet loses precision.

In Web Tidy's 14-column output, column A is id, C is username, J is audience, K is sourceProfile, M is collectedAt, and N is rowIndex. The formulas below use O for accountKey and P for review status.

Step 1: keep Raw and create a Working copy#

Do not edit downloaded CSV/XLSX files directly. Copy each file into a Raw_* worksheet and append the rows in Working. Add source filename, source sheet, and original row number so every retained row can be traced back.

Step 2: import IDs as text#

When importing CSV, set column A to text. If IDs already appear in scientific notation, changing the cell format afterward may not recover lost digits. Re-import from the original.

Step 3: create an ID-first accountKey#

In O2:

Copy=IF(A2<>"","id:"&A2,"user:"&LOWER(IF(LEFT(TRIM(C2),1)="@",MID(TRIM(C2),2,99),TRIM(C2))))

This produces id:<id> when A2 is available. Otherwise, it trims C2, removes one leading @, lowercases the username, and produces user:<username>.

If both ID and username are empty, label the row Missing key. Do not collapse every empty key into one supposed account.

Step 4: flag first and repeated appearances#

In P2:

Copy=IF(OR(A2<>"",C2<>""),IF(COUNTIF($O$2:O2,O2)=1,"Keep first","Review duplicate"),"Missing key")

To count all appearances of a key, use Q2:

Copy=COUNTIF($O:$O,O2)

Fill the formulas down and filter Review duplicate before deleting anything. Microsoft recommends copying the original range and reviewing unique/duplicate values before permanent removal.

Source: Microsoft Support — Filter for or remove duplicate values and Count how often a value occurs.

Step 5: review conflicts before removal#

For every repeated key, check:

• whether IDs are exactly equal;
• whether different usernames are observations attached to the same ID;
• which sourceProfile, audience, and collectedAt values need to survive;
• which row has more complete optional fields;
• which conflicts require manual review.

After review, use Data → Remove Duplicates in the working copy and select only the accountKey column. If you intend to keep the latest observation, sort by collectedAt first and document that rule.

XLOOKUP can map a retained key back to an original row and return fields from that row. Microsoft documents exact match as the default, but XLOOKUP is unavailable in Excel 2016 and Excel 2019, so it is not required for this workflow.

Source: Microsoft Support — XLOOKUP.

Step 6: keep duplicate accounts across different sources#

If you want to deduplicate only inside each source profile and audience direction, create a scoped key:

Copy=LOWER(TRIM(K2))&"|"&LOWER(TRIM(J2))&"|"&O2

The same account will remain as separate rows when it appears under different source profiles or in both Followers and Following. That is correct for source-level observations, but not for a global unique-account count.

Reconcile row counts#

Before removal, record:

CopyrawRows = keptRows + duplicateRows + missingKeyRows

After removal, manually inspect 5–10 duplicate groups and confirm:

1. Long IDs were not truncated.
2. The same ID with a different username was not split.
3. Different IDs with the same username were not silently merged.
4. Required source and time provenance was moved to a summary or provenance sheet.
5. Original files remain unchanged.

False-positive and false-merge boundaries#

• Missing ID: username fallback can normalize formatting but cannot prove that two rows across time are the same account.
• Username rename: equal ID with different usernames should share a key, but the display value still needs a documented time rule.
• Username reuse: the same username at different times may not represent the same account.
• Different IDs, same username: keep both rows and review; do not auto-merge.
• Cross-source appearance: appearing in two profiles' Followers lists is not an error. Whether to merge depends on account-level union versus source-level analysis.
• Bounded inputs: 100, 200, 500, or 1,000-row ceilings and partial tasks constrain coverage. Deduplication cannot turn a bounded subset into a complete list.
• Permission: appearance in a follower file does not establish identity, interest, marketing consent, or permission to contact.

Frequently asked questions#

Why prefer ID over username?#

ID is a better stable match key, while username can change. Import long IDs as text so the spreadsheet does not corrupt the key first.

Does one Web Tidy export need another deduplication pass?#

The product deduplicates by id by default. The spreadsheet workflow is for combining multiple files, dates, or sources.

Can I deduplicate only by username?#

Use username only as a fallback when ID is missing, and keep the result marked as lower confidence.

Does deduplication reveal unfollowers?#

No. Historical change requires two comparable Followers snapshots and separate coverage review.

Use the cleaned data for the right comparison#

• Current Following versus Followers: Who Doesn't Follow You Back on Instagram.
• Followers at two dates: How to See Who Unfollowed You Using Two Exports.
• Followers from two profiles: Common Followers Between Two Instagram Accounts.
• Export method basics: How to Export Instagram Followers.

Open Instagram Follower Exporter