Skip to content

Commit b0f0c61

Browse files
gantesumedhghaisas2SunMarczucchini-nlp
authored
Add SynthID (watermerking by Google DeepMind) (#34350)
* Add SynthIDTextWatermarkLogitsProcessor * esolving comments. * Resolving comments. * esolving commits, * Improving SynthIDWatermark tests. * switch to PT version * detector as pretrained model + style * update training + style * rebase * Update logits_process.py * Improving SynthIDWatermark tests. * Shift detector training to wikitext negatives and stabilize with lower learning rate. * Clean up. * in for 7B * cleanup * upport python 3.8. * README and final cleanup. * HF Hub upload and initiaze. * Update requirements for synthid_text. * Adding SynthIDTextWatermarkDetector. * Detector testing. * Documentation changes. * Copyrights fix. * Fix detector api. * ironing out errors * ironing out errors * training checks * make fixup and make fix-copies * docstrings and add to docs * copyright * BC * test docstrings * move import * protect type hints * top level imports * watermarking example * direct imports * tpr fpr meaning * process_kwargs * SynthIDTextWatermarkingConfig docstring * assert -> exception * example updates * no immutable dict (cant be serialized) * pack fn * einsum equivalent * import order * fix test on gpu * add detector example --------- Co-authored-by: Sumedh Ghaisas <sumedhg@google.com> Co-authored-by: Marc Sun <marc@huggingface.co> Co-authored-by: sumedhghaisas2 <138781311+sumedhghaisas2@users.noreply.github.com> Co-authored-by: raushan <raushan@huggingface.co>
1 parent e50bf61 commit b0f0c61

File tree

15 files changed

+2238
-80
lines changed

15 files changed

+2238
-80
lines changed

docs/source/en/internal/generation_utils.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,9 @@ generation.
185185
[[autodoc]] SuppressTokensLogitsProcessor
186186
- __call__
187187

188+
[[autodoc]] SynthIDTextWatermarkLogitsProcessor
189+
- __call__
190+
188191
[[autodoc]] TemperatureLogitsWarper
189192
- __call__
190193

@@ -418,5 +421,20 @@ A [`Constraint`] can be used to force the generation to include specific tokens
418421

419422
## Watermark Utils
420423

424+
[[autodoc]] WatermarkingConfig
425+
- __call__
426+
421427
[[autodoc]] WatermarkDetector
422428
- __call__
429+
430+
[[autodoc]] BayesianDetectorConfig
431+
- __call__
432+
433+
[[autodoc]] BayesianDetectorModel
434+
- __call__
435+
436+
[[autodoc]] SynthIDTextWatermarkingConfig
437+
- __call__
438+
439+
[[autodoc]] SynthIDTextWatermarkDetector
440+
- __call__

docs/source/en/main_classes/text_generation.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,6 @@ like token streaming.
4141
- validate
4242
- get_generation_mode
4343

44-
[[autodoc]] generation.WatermarkingConfig
45-
4644
## GenerationMixin
4745

4846
[[autodoc]] GenerationMixin
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# SynthID Text
2+
3+
This project showcases the use of SynthIDText for watermarking LLMs. The code shown in this repo also
4+
demostrates the training of the detector for detecting such watermarked text. This detector can be uploaded onto
5+
a private HF hub repo (private for security reasons) and can be initialized again through pretrained model loading also shown in this script.
6+
7+
See our blog post: https://huggingface.co/blog/synthid-text
8+
9+
10+
## Python version
11+
12+
User would need python 3.9 to run this example.
13+
14+
## Installation and running
15+
16+
Once you install transformers you would need to install requirements for this project through requirements.txt provided in this folder.
17+
18+
```
19+
pip install -r requirements.txt
20+
```
21+
22+
## To run the detector training
23+
24+
```
25+
python detector_training.py --model_name=google/gemma-7b-it
26+
```
27+
28+
Check the script for more parameters are are tunable and check out paper at link
29+
https://www.nature.com/articles/s41586-024-08025-4 for more information on these parameters.
30+
31+
## Caveat
32+
33+
Make sure to run the training of the detector and the detection on the same hardware
34+
CPU, GPU or TPU to get consistent results (we use detecterministic randomness which is hardware dependent).

0 commit comments

Comments
 (0)