view bin/percent_encode.py @ 115:0b1e6e134aca

robotstxt and crawldiagnostics get free ride, get rid of DFQ and xq, big simplification and refactor as a result, fix bug in date stream eof handling
author Henry S. Thompson <ht@inf.ed.ac.uk>
date Tue, 26 Sep 2023 17:42:57 +0100
parents 1daa8e444cfe
children
line wrap: on
line source

'''Handle unquoting of non-UTF-8 bytes by %-encoding them'''
import codecs

def percent_encode(ude):
  #print(ude.object,ude.object[ude.start:ude.end])
  return (''.join('%%%X'%c for c in ude.object[ude.start:ude.end]),
          ude.end)

codecs.register_error('percent',percent_encode)