defusedxml · pypi · 2019-09-16 · synopsis the results of an ... long time with a small to...
Post on 19-Apr-2020
3 Views
Preview:
TRANSCRIPT
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 1/24
Navigation
� Projectdescription
ś Releasehistory
Ñ Downloadfiles
Projectlinks
ŝ Homepage
� Download
Statistics
Projectdescription
“It’sjustXML,whatcouldprobablygowrong?”
ChristianHeimes<christian@python.org >
Synopsis
TheresultsofanattackonavulnerableXMLlibrarycanbefairlydramatic.WithjustafewhundredBytesofXMLdataanattackercanoccupyseveralGigabytesofmemorywithinseconds.AnattackercanalsokeepCPUsbusyforalongtimewithasmalltomediumsizerequest.Undersomecircumstancesitisevenpossibletoaccesslocalfilesonyourserver,tocircumventafirewall,ortoabuseservicestoreboundattackstothirdparties.
TheattacksuseandabuselesscommonfeaturesofXMLanditsparsers.Themajorityofdevelopersareunacquaintedwithfeatures
Help Donate Login Register
pipinstalldefusedxml==0.5.0
defusedxml0.5.0 âNewerversionavailable(0.6.0)
Lastreleased:Feb9,2017
XMLbombprotectionforPythonstdlibmodules
Searchprojects ȡ
®
*
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 2/24
GitHubstatistics:
ɕ Stars:126
� Forks:25
á Openissues/PRs:8
ViewstatisticsforthisprojectviaLibraries.io ,orbyusingGoogleBigQuery
Meta
License:PythonSo�wareFoundationLicense(PSFL)
Author:ChristianHeimes
ɶxml,bomb,DoS
Maintainers
tiran
Classifiers
DevelopmentStatus5-Production/Stable
IntendedAudienceDevelopers
LicenseOSIApproved::PythonSo�ware
suchasprocessinginstructionsandentityexpansionsthatXMLinheritedfromSGML.Atbesttheyknowabout <!DOCTYPE> fromexperiencewithHTMLbuttheyarenotawarethatadocumenttypedefinition(DTD)cangenerateanHTTPrequestorloadafilefromthefilesystem.
Noneoftheissuesisnew.Theyhavebeenknownforalongtime.Billionlaughswasfirstreportedin2003.NeverthelesssomeXMLlibrariesandapplicationsarestillvulnerableandevenheavyusersofXMLaresurprisedbythesefeatures.It’shardtosaywhomtoblameforthesituation.It’stooshortsightedtoshi�allblameonXMLparsersandXMLlibrariesforusinginsecuredefaultsettings.A�eralltheyproperlyimplementXMLspecifications.Applicationdevelopersmustnotrelythatalibraryisalwaysconfiguredforsecurityandpotentialharmfuldatabydefault.
TableofContents
Synopsis
Attackvectors
billionlaughs/exponentialentityexpansion
quadraticblowupentityexpansion
externalentityexpansion(remote)
externalentityexpansion(localfile)
DTDretrieval
PythonXMLLibraries
Settingsinstandardlibrary
defusedxml
defusedxml(package)
defusedxml.cElementTree
defusedxml.ElementTree
defusedxml.expatreader
defusedxml.sax
defusedxml.expatbuilder
defusedxml.minidom
defusedxml.pulldom
defusedxml.xmlrpc
æ
æ
æ
æ
æ
*
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 3/24
FoundationLicense
NaturalLanguageEnglish
ProgrammingLanguagePythonPython::2Python::2.7Python::3Python::3.4Python::3.5Python::3.6
TopicTextProcessing::Markup::XML
defusedxml.lxml
defusedexpat
Modificationsinexpat
HowtoavoidXMLvulnerabilities
Otherthingstoconsider
attributeblowup/hashcollisionattack
decompressionbomb
ProcessingInstruction
OtherDTDfeatures
XPath
XPathinjectionattacks
XInclude
XMLSchemalocation
XSLTransformation
RelatedCVEs
Otherlanguages/frameworks
Perl
Ruby
PHP
C#/.NET/Mono
Java
TODO
License
Acknowledgements
References
Changelog
defusedxml0.5.0
defusedxml0.5.0.rc1
defusedxml0.4.1
defusedxml0.4
defusedxml0.3
defusedxml0.2
defusedxml0.1
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 4/24
Attackvectors
billionlaughs/exponentialentityexpansion
TheBillionLaughsattack–alsoknownasexponentialentityexpansion–usesmultiplelevelsofnestedentities.Theoriginalexampleuses9levelsof10expansionsineachleveltoexpandthestring lol toastringof3*10 bytes,hencethename“billionlaughs”.Theresultingstringoccupies3GB(2.79GiB)ofmemory;intermediatestringsrequireadditionalmemory.Becausemostparsersdon’tcachetheintermediatestepforeveryexpansionitisrepeatedoverandoveragain.ItincreasestheCPUloadevenmore.
AnXMLdocumentofjustafewhundredbytescandisruptallservicesonamachinewithinseconds.
ExampleXML:
<!DOCTYPExmlbomb[<!ENTITYa"1234567890"><!ENTITYb"&a;&a;&a;&a;&a;&a;&a;&a;"><!ENTITYc"&b;&b;&b;&b;&b;&b;&b;&b;"><!ENTITYd"&c;&c;&c;&c;&c;&c;&c;&c;">]><bomb>&d;</bomb>
quadraticblowupentityexpansion
AquadraticblowupattackissimilartoaBillionLaughsattack;itabusesentityexpansion,too.Insteadofnestedentitiesitrepeatsonelargeentitywithacoupleofthousandcharsoverandoveragain.Theattackisn’tase�icientastheexponentialcasebutitavoidstriggeringcountermeasuresofparsersagainstheavilynestedentities.SomeparserslimitthedepthandbreadthofasingleentitybutnotthetotalamountofexpandedtextthroughoutanentireXMLdocument.
9
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 5/24
Amedium-sizedXMLdocumentwithacoupleofhundredkilobytescanrequireacoupleofhundredMBtoseveralGBofmemory.Whentheattackiscombinedwithsomelevelofnestedexpansionanattackerisabletoachieveahigherratioofsuccess.
externalentityexpansion(remote)
Entitydeclarationscancontainmorethanjusttextforreplacement.Theycanalsopointtoexternalresourcesbypublicidentifiersorsystemidentifiers.SystemidentifiersarestandardURIs.WhentheURIisaURL(e.g.a http:// locator)someparsersdownloadtheresourcefromtheremotelocationandembedthemintotheXMLdocumentverbatim.
Simpleexampleofaparsedexternalentity:
ThecaseofparsedexternalentitiesworksonlyforvalidXMLcontent.TheXMLstandardalsosupportsunparsedexternalentitieswitha NDatadeclaration .
Externalentityexpansionopensthedoortoplentyofexploits.AnattackercanabuseavulnerableXMLlibraryandapplicationtoreboundandforwardnetworkrequestswiththeIPaddressoftheserver.Ithighlydependsontheparserandtheapplicationwhatkindofexploitispossible.Forexample:
Anattackercancircumventfirewallsandgainaccesstorestrictedresourcesasalltherequestsaremadefroman
<!DOCTYPEbomb[<!ENTITYa"xxxxxxx...acoupleoftenthousandchars]><bomb>&a;&a;&a;...repeat</bomb>
<!DOCTYPEexternal[<!ENTITYeeSYSTEM"http://www.python.org/some.xml">]><root>ⅇ</root>
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 6/24
internalandtrustworthyIPaddress,notfromtheoutside.
Anattackercanabuseaservicetoattack,spyonorDoSyourserversbutalsothirdpartyservices.TheattackisdisguisedwiththeIPaddressoftheserverandtheattackerisabletoutilizethehighbandwidthofabigmachine.
Anattackercanexhaustadditionalresourcesonthemachine,e.g.withrequeststoaservicethatdoesn’trespondorrespondswithverylargefiles.
Anattackermaygainknowledge,when,howo�enandfromwhichIPaddressaXMLdocumentisaccessed.
AnattackercouldsendmailfrominsideyournetworkiftheURLhandlersupports smtp:// URIs.
externalentityexpansion(localfile)
Externalentitieswithreferencestolocalfilesareasub-caseofexternalentityexpansion.It’slistedasanextraattackbecauseitdeservesextraattention.SomeXMLlibrariessuchaslxmldisablenetworkaccessbydefaultbutstillallowentityexpansionwithlocalfileaccessbydefault.Localfilesareeitherreferencedwithafile:// URLorbyafilepath(eitherrelativeorabsolute).
Anattackermaybeabletoaccessanddownloadallfilesthatcanbereadbytheapplicationprocess.Thismayincludecriticalconfigurationfiles,too.
<!DOCTYPEexternal[<!ENTITYeeSYSTEM"file:///PATH/TO/simple.xml">]><root>ⅇ</root>
DTDretrieval
Thiscaseissimilartoexternalentityexpansion,too.SomeXMLlibrarieslikePython’sxml.dom.pulldomretrievedocumenttypedefinitionsfromremoteorlocallocations.Severalattackscenariosfromtheexternalentitycaseapplytothisissueaswell.
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 7/24
PythonXMLLibraries
vulnerabilitiesandfeatures
kind sax etree minidom pulldom xmlrpc lxml genshi
billionlaughs
True True True True TrueFalse(1)
False(5)
quadraticblowup
True True True True True TrueFalse(5)
externalentityexpansion(remote)
TrueFalse(3)
False(4) True falseFalse(1)
False(5)
externalentityexpansion(localfile)
TrueFalse(3)
False(4) True false TrueFalse(5)
DTDretrieval
True False False True falseFalse(1)
False
gzipbomb
False False False False Truepartly(2)
False
xpathsupport(7)
False False False False False True False
xsl(t)support(7)
False False False False False True False
<?xmlversion="1.0"encoding="utf-8"?><!DOCTYPEhtmlPUBLIC"-//W3C//DTDXHTML1.0Transitio"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitiona<html><head/><body>text</body></html>
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 8/24
kind sax etree minidom pulldom xmlrpc lxml genshi
xincludesupport(7)
FalseTrue(6)
False False FalseTrue(6)
True
Clibrary expat expat expat expat expat libxml2 expat
1.Lxmlisprotectedagainstbillionlaughsattacksanddoesn’tdonetworklookupsbydefault.
2.libxml2andlxmlarenotdirectlyvulnerabletogzipdecompressionbombsbuttheydon’tprotectyouagainstthemeither.
3.xml.etreedoesn’texpandentitiesandraisesaParserErrorwhenanentityoccurs.
4.minidomdoesn’texpandentitiesandsimplyreturnstheunexpandedentityverbatim.
5.genshi.inputofgenshi0.6doesn’tsupportentityexpansionandraisesaParserErrorwhenanentityoccurs.
6.Libraryhas(limited)XIncludesupportbutrequiresanadditionalsteptoprocessinclusion.
7.Thesearefeaturesbuttheymayintroduceexploitableholes,seeOtherthingstoconsider
Settingsinstandardlibrary
xml.sax.handlerFeatures
feature_external_ges(http://xml.org/sax/features/external-general-entities)
disablesexternalentityexpansion
feature_external_pes(http://xml.org/sax/features/external-parameter-entities)
theoptionisignoredanddoesn’tmodifyanyfunctionality
DOMxml.dom.xmlbuilder.Options
external_parameter_entities
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 9/24
ignored
external_general_entitiesignored
external_dtd_subsetignored
entitiesunsure
defusedxml
Thedefusedxmlpackage(defusedxmlonPyPI)containsseveralPython-onlyworkaroundsandfixesfordenialofserviceandothervulnerabilitiesinPython’sXMLlibraries.Inordertobenefitfromtheprotectionyoujusthavetoimportandusethelistedfunctions/classesfromtherightdefusedxmlmoduleinsteadoftheoriginalmodule.Merelydefusedxml.xmlrpcisimplementedasmonkeypatch.
Insteadof:
>>>fromxml.etree.ElementTreeimportparse>>>et=parse(xmlfile)
altercodeto:
>>>fromdefusedxml.ElementTreeimportparse>>>et=parse(xmlfile)
Additionallythepackagehasanuntestedfunctiontomonkeypatchallstdlibmoduleswith defusedxml.defuse_stdlib() .
Allfunctionsandparserclassesacceptthreeadditionalkeywordarguments.Theyreturneitherthesameobjectsastheoriginalfunctionsorcompatiblesubclasses.
forbid_dtd(default:False)
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 10/24
disallowXMLwitha <!DOCTYPE> processinginstructionandraiseaDTDForbiddenexceptionwhenaDTDprocessinginstructionisfound.
forbid_entities(default:True)disallowXMLwith <!ENTITY> declarationsinsidetheDTDandraiseanEntitiesForbiddenexceptionwhenanentityisdeclared.
forbid_external(default:True)disallowanyaccesstoremoteorlocalresourcesinexternalentitiesorDTDandraisinganExternalReferenceForbiddenexceptionwhenaDTDorentityreferencesanexternalresource.
defusedxml(package)
DefusedXmlException,DTDForbidden,EntitiesForbidden,ExternalReferenceForbidden,NotSupportedError
defuse_stdlib()(experimental)
defusedxml.cElementTree
parse(),iterparse(),fromstring(),XMLParser
defusedxml.ElementTree
parse(),iterparse(),fromstring(),XMLParser
defusedxml.expatreader
create_parser(),DefusedExpatParser
defusedxml.sax
parse(),parseString(),create_parser()
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 11/24
defusedxml.expatbuilder
parse(),parseString(),DefusedExpatBuilder,DefusedExpatBuilderNS
defusedxml.minidom
parse(),parseString()
defusedxml.pulldom
parse(),parseString()
defusedxml.xmlrpc
Thefixisimplementedasmonkeypatchforthestdlib’sxmlrpcpackage(3.x)orxmlrpclibmodule(2.x).Thefunctionmonkey_patch()enablesthefixes,unmonkey_patch()removesthepatchandputsthecodeinitsformerstate.
ThemonkeypatchprotectsagainstXMLrelatedattacksaswellasdecompressionbombsandexcessivelylargerequestsorresponses.Thedefaultsettingis30MBforrequests,responsesandgzipdecompression.YoucanmodifythedefaultbychangingthemodulevariableMAX_DATA.Avalueof-1disablesthelimit.
defusedxml.lxml
Themoduleactsasanexamplehowyoucouldprotectcodethatuseslxml.etree.ItimplementsacustomElementclassthatfiltersoutEntityinstances,acustomparserfactoryandathreadlocalstorageforparserinstances.Italsohasacheck_docinfo()functionwhichinspectsatreeforinternalorexternalDTDsandentitydeclarations.Inordertocheckforentitieslxml>3.0isrequired.
parse(),fromstring()RestrictedElement,GlobalParserTLS,getDefaultParser(),check_docinfo()
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 12/24
defusedexpat
Thedefusedexpatpackage(defusedexpatonPyPI)comeswithbinaryextensionsandamodifiedexpatlibaryinsteadofthestandardexpatparser.It’sbasicallyastand-aloneversionofthepatchesforPython’sstandardlibraryCextensions.
Modificationsinexpat
newdefinitions:
XML_BOMB_PROTECTIONXML_DEFAULT_MAX_ENTITY_INDIRECTIONSXML_DEFAULT_MAX_ENTITY_EXPANSIONSXML_DEFAULT_RESET_DTD
newXML_FeatureEnummembers:
XML_FEATURE_MAX_ENTITY_INDIRECTIONSXML_FEATURE_MAX_ENTITY_EXPANSIONSXML_FEATURE_IGNORE_DTD
newXML_Errormembers:
XML_ERROR_ENTITY_INDIRECTIONSXML_ERROR_ENTITY_EXPANSION
newAPIfunctions:
intXML_GetFeature(XML_Parserparser,enumXML_FeatureEnumfeature,long*value);intXML_SetFeature(XML_Parserparser,enumXML_FeatureEnumfeature,longvalue);intXML_GetFeatureDefault(enumXML_FeatureEnumfeature
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 13/24
XML_FEATURE_MAX_ENTITY_INDIRECTIONS
Limittheamountofindirectionsthatareallowedtooccurduringtheexpansionofanestedentity.Acounterstartswhenanentityreferenceisencountered.Itresetsa�ertheentityisfullyexpanded.Thelimitprotectstheparseragainstexponentialentityexpansionattacks(akabillionlaughsattack).WhenthelimitisexceededtheparserstopsandfailswithXML_ERROR_ENTITY_INDIRECTIONS.Avalueof0disablestheprotection.
Supportedrange0..UINT_MAX
Default40
XML_FEATURE_MAX_ENTITY_EXPANSIONS
Limitthetotallengthofallentityexpansionsthroughouttheentiredocument.Thelengthsofallentitiesareaccumulatedinaparservariable.Thesettingprotectsagainstquadraticblowupattacks(lotsofexpansionsofalargeentitydeclaration).Whenthesumofallentitiesexceedsthelimit,theparserstopsandfailswithXML_ERROR_ENTITY_EXPANSION.Avalueof0disablestheprotection.
Supportedrange0..UINT_MAX
Default8MiB
XML_FEATURE_RESET_DTD
ResetallDTDinformationa�erthe<!DOCTYPE>blockhasbeenparsed.Whentheflagisset(default:false)allDTDinformationa�ertheendDoctypeDeclHandlerhasbeencalled.TheflagcanbesetinsidetheendDoctypeDeclHandler.WithoutDTD
long*value);intXML_SetFeatureDefault(enumXML_FeatureEnumfeaturelongvalue);
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 14/24
informationanyentityreferenceinthedocumentbodyleadstoXML_ERROR_UNDEFINED_ENTITY.
Supportedrange0,1
Default0
HowtoavoidXMLvulnerabilities
Bestpractices
Don’tallowDTDs
Don’texpandentities
Don’tresolveexternals
Limitparsedepth
Limittotalinputsize
Limitparsetime
FavoraSAXoriterparse-likeparserforpotentiallargedata
ValidateandproperlyquoteargumentstoXSLtransformationsandXPathqueries
Don’tuseXPathexpressionfromuntrustedsources
Don’tapplyXSLtransformationsthatcomeuntrustedsources
(basedonBradHill’sAttackingXMLSecurity)
Otherthingstoconsider
XML,XMLparsersandprocessinglibrarieshavemorefeaturesandpossibleissuethatcouldleadtoDoSvulnerabilitiesorsecurityexploitsinapplications.Ihavecompiledanincompletelistoftheoreticalissuesthatneedfurtherresearchandmoreattention.Thelistisdeliberatelypessimisticandabitparanoid,too.Itcontainsthingsthatmightgowrongunderda�ycircumstances.
attributeblowup/hashcollisionattack
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 15/24
XMLparsersmayuseanalgorithmwithquadraticruntimeO(n )tohandleattributesandnamespaces.Ifituseshashtables(dictionaries)tostoreattributesandnamespacestheimplementationmaybevulnerabletohashcollisionattacks,thusreducingtheperformancetoO(n )again.IneithercaseanattackerisabletoforgeadenialofserviceattackwithanXMLdocumentthatcontainsthousandsuponthousandsofattributesinasinglenode.
Ihaven’tresearchedyetifexpat,pyexpatorlibxml2arevulnerable.
decompressionbomb
Theissueofdecompressionbombs(akaZIPbomb)applytoallXMLlibrariesthatcanparsecompressedXMLstreamlikegzippedHTTPstreamsorLZMA-edfiles.Foranattackeritcanreducetheamountoftransmitteddatabythreemagnitudesormore.Gzipisabletocompress1GiBzerostoroughly1MB,lzmaisevenbetter:
NoneofPython’sstandardXMLlibrariesdecompressstreamsexceptfor xmlrpclib .Themoduleisvulnerable<http://bugs.python.org/issue16043>todecompressionbombs.
lxmlcanloadandprocesscompresseddatathroughlibxml2transparently.libxml2canhandleevenverylargeblobsofcompresseddatae�icientlywithoutusingtoomuchmemory.Butitdoesn’tprotectapplicationsfromdecompressionbombs.AcarefullywrittenSAXoriterparse-likeapproachcanbesafe.
ProcessingInstruction
PI’slike:
$ddif=/dev/zerobs=1Mcount=1024|gzip>zeros.gz$ddif=/dev/zerobs=1Mcount=1024|lzma-z>zeros.x$ls-shzeros.*1020Kzeros.gz148Kzeros.xy
2
2
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 16/24
<?xml-stylesheettype="text/xsl"href="style.xsl"?>
mayimposemorethreatsforXMLprocessing.Itdependsifandhowaprocessorhandlesprocessinginstructions.TheissueofURLretrievalwithnetworkorlocalfileaccessapplytoprocessinginstructions,too.
OtherDTDfeatures
DTDhasmorefeatureslike <!NOTATION> .Ihaven’tresearchedhowthesefeaturesmaybeasecuritythreat.
XPath
XPathstatementsmayintroduceDoSvulnerabilities.Codeshouldneverexecutequeriesfromuntrustedsources.AnattackermayalsobeabletocreateaXMLdocumentthatmakescertainXPathqueriescostlyorresourcehungry.
XPathinjectionattacks
XPathinjecitonattacksprettymuchworklikeSQLinjectionattacks.ArgumentstoXPathqueriesmustbequotedandvalidatedproperly,especiallywhentheyaretakenfromtheuser.ThepageAvoidthedangersofXPathinjectionlistsomeramificationsofXPathinjections.
Python’sstandardlibrarydoesn’thaveXPathsupport.LxmlsupportsparameterizedXPathquerieswhichdoesproperquoting.Youjusthavetouseitsxpath()methodcorrectly:
#DON'T>>>tree.xpath("/tag[@id='%s']"%value)#insteaddo>>>tree.xpath("/tag[@id=$tagid]",tagid=name)
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 17/24
XInclude
XMLInclusionisanotherwaytoloadandincludeexternalfiles:
<rootxmlns:xi="http://www.w3.org/2001/XInclude"><xi:includehref="filename.txt"parse="text"/></root>
ThisfeatureshouldbedisabledwhenXMLfilesfromanuntrustedsourceareprocessed.SomePythonXMLlibrariesandlibxml2supportXIncludebutdon’thaveanoptiontosandboxinclusionandlimitittoalloweddirectories.
XMLSchemalocation
AvalidatingXMLparsermaydownloadschemafilesfromtheinformationina xsi:schemaLocation attribute.
XSLTransformation
YoushouldkeepinmindthatXSLTisaTuringcompletelanguage.NeverprocessXSLTcodefromunknownoruntrustedsource!XSLTprocessorsmayallowyoutointeractwithexternalresourcesinwaysyoucan’tevenimagine.Someprocessorsevensupportextensionsthatallowread/writeaccesstofilesystem,accesstoJREobjectsorscriptingwithJython.
ExamplefromAttackingXMLSecurityforXalan-J:
<eadxmlns="urn:isbn:1-931666-22-9"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instxsi:schemaLocation="urn:isbn:1-931666-22-9http:</ead>
<xsl:stylesheetversion="1.0"xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 18/24
RelatedCVEs
CVE-2013-1664UnrestrictedentityexpansioninducesDoSvulnerabilitiesinPythonXMLlibraries(XMLbomb)
CVE-2013-1665ExternalentityexpansioninPythonXMLlibrariesinflictspotentialsecurityflawsandDoSvulnerabilities
Otherlanguages/frameworks
Severalotherprogramminglanguagesandframeworksarevulnerableaswell.Acoupleofthemarea�ectedbythefactthatlibxml2upto2.9.0hasnoprotectionagainstquadraticblowupattacks.Mostofthemhavepotentialdangerousdefaultsettingsforentityexpansionandexternalentities,too.
Perl
Perl’sXML::Simpleisvulnerabletoquadraticentityexpansionandexternalentityexpansion(bothlocalandremote).
Ruby
Ruby’sREXMLdocumentparserisvulnerabletoentityexpansionattacks(bothquadraticandexponential)butitdoesn’tdoexternal
xmlns:rt="http://xml.apache.org/xalan/java/java.langxmlns:ob="http://xml.apache.org/xalan/java/java.langexclude-result-prefixes="rtob"><xsl:templatematch="/"><xsl:variablename="runtimeObject"select="rt:getR<xsl:variablename="command"select="rt:exec($runtimeObject,'c:\Windows<xsl:variablename="commandAsString"select="ob:toS<xsl:value-ofselect="$commandAsString"/></xsl:template></xsl:stylesheet>
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 19/24
entityexpansionbydefault.Inordertocounteractentityexpansionyouhavetodisablethefeature:
REXML::Document.entity_expansion_limit=0
libxml-rubyandhpricotdon’texpandentitiesintheirdefaultconfiguration.
PHP
PHP’sSimpleXMLAPIisvulnerabletoquadraticentityexpansionandloadsentitesfromlocalandremoteresources.TheoptionLIBXML_NONET disablesnetworkaccessbutstillallowslocalfileaccess. LIBXML_NOENT seemstohavenoe�ectonentityexpansioninPHP5.4.6.
C#/.NET/Mono
InformationinXMLDoSandDefenses(MSDN)suggestthat.NETisvulnerablewithitsdefaultsettings.ThearticlecontainscodesnippetshowtocreateasecureXMLreader:
Java
Untested.ThedocumentationofXercesanditsXercesSecurityManangersoundslikeXercesisalsovulnerabletobillionlaughattackswithitsdefaultsettings.Italsodoesentityresolvingwhenan org.xml.sax.EntityResolver isconfigured.I’mnotyetsureaboutthedefaultsettinghere.
XmlReaderSettingssettings=newXmlReaderSettings();settings.ProhibitDtd=false;settings.MaxCharactersFromEntities=1024;settings.XmlResolver=null;XmlReaderreader=XmlReader.Create(stream,settings)
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 20/24
Javaspecialistssuggesttohaveacustombuilderfactory:
TODO
DOM:Usexml.dom.xmlbuilderoptionsforentityhandling
SAX:takefeature_external_gesandfeature_external_pes(?)intoaccount
testexperimentalmonkeypatchingofstdlibmodules
improvedocumentation
License
Copyright(c)2013-2017byChristianHeimes<christian@python.org >
LicensedtoPSFunderaContributorAgreement.
Seehttp://www.python.org/psf/licenseforlicensingdetails.
Acknowledgements
BrettCannon(PythonCoredeveloper)reviewandcodecleanup
AntoinePitrou(PythonCoredeveloper)codereview
DocumentBuilderFactorybuilderFactory=DocumentBuildebuilderFactory.setXIncludeAware(False);builderFactory.setExpandEntityReferences(False);builderFactory.setFeature(XMLConstants.FEATURE_SECURE_#eitherbuilderFactory.setFeature("http://apache.org/xml/feat#orifyouneedDTDsbuilderFactory.setFeature("http://xml.org/sax/featuresbuilderFactory.setFeature("http://xml.org/sax/featuresbuilderFactory.setFeature("http://apache.org/xml/featbuilderFactory.setFeature("http://apache.org/xml/feat
*
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 21/24
AaronPatterson,BenMurphyandMichaelKoziarski(Rubycommunity)
ManythankstoAaron,BenandMichaelfromtheRubycommunityfortheirreportandassistance.
ThierryCarrez(OpenStack)ManythankstoThierryforhisreporttothePythonSecurityResponseTeamonbehalfoftheOpenStacksecurityteam.
CarlMeyer(Django)ManythankstoCarlforhisreporttoPSRTonbehalfoftheDjangosecurityteam.
DanielVeillard(libxml2)ManythankstoDanielforhisinsightandassistancewithlibxml2.
semanticsGmbH(http://www.semantics.de/)Manythankstomyemployersemanticsforlettingmeworkontheissueduringworkinghoursaspartofsemantics’sopensourceinitiative.
References
XMLDoSandDefenses(MSDN)
BillionLaughsonWikipedia
ZIPbombonWikipedia
ConfigureSAXparsersforsecureprocessing
TestingforXMLInjection
Changelog
defusedxml0.5.0
Releasedate:07-Feb-2017
Nochanges
defusedxml0.5.0.rc1
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 22/24
Releasedate:28-Jan-2017
AddcompatibilitywithPython3.6
DropsupportforPython2.6,3.1,3.2,3.3
Fixlxmltests(XMLSyntaxError:Detectedanentityreferenceloop)
defusedxml0.4.1
Releasedate:28-Mar-2013
Addmoredemoexploits,e.g.python_external.pyandXalanXSLTdemos.
Improveddocumentation.
defusedxml0.4
Releasedate:25-Feb-2013
Asperhttp://seclists.org/oss-sec/2013/q1/340pleaseREJECTCVE-2013-0278,CVE-2013-0279andCVE-2013-0280anduseCVE-2013-1664,CVE-2013-1665forOpenStack/etc.
Addmissingparser_listargumenttosax.make_parser().Theargumentisignored,though.(thankstoFlorianApolloner)
AdddemoexploitforexternalentityattackonPython’sSAXparser,XML-RPCandWebDAV.
defusedxml0.3
Releasedate:19-Feb-2013
Improvedocumentation
defusedxml0.2
Releasedate:15-Feb-2013
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 23/24
RenameExternalEntitiesForbiddentoExternalReferenceForbidden
Renamedefusedxml.lxml.check_dtd()tocheck_docinfo()
Unifyargumentnamesincallbacks
Addargumentsandformattedrepresentationtoexceptions
Addforbid_externalargumenttoallfunctionsandclasss
Moretests
LOTSofdocumentation
Addexamplecodeforotherlanguages(Ruby,Perl,PHP)andparsers(Genshi)
AddprotectionagainstXMLandgzipattackstoxmlrpclib
defusedxml0.1
Releasedate:08-Feb-2013
InitialandinternalreleaseforPSRTreview
Help
Installingpackages
Uploadingpackages
Userguide
FAQs
AboutPyPI
PyPIonTwitter
Infrastructuredashboard
Packageindexnameretention
Oursponsors
ContributingtoPyPI
Bugsandfeedback
ContributeonGitHub
UsingPyPI
Codeofconduct
Reportsecurityissue
æ
æ
æ
æ
æ
æ
æ
æ
8/5/2019 defusedxml · PyPI
https://pypi.org/project/defusedxml/0.5.0/ 24/24
Developmentcredits Privacypolicy
Termsofuse
Status:AllSystemsOperational
DevelopedandmaintainedbythePythoncommunity,forthePythoncommunity.Donatetoday!
©2019PythonSo�wareFoundation
æ æ
æ
æ
Desktopversion
ElasticSearch
PingdomMonitoring
GoogleBigQuery
SentryErrorlogging
AWSCloudcomputing
DataDogMonitoring
FastlyCDN
SignalFxSupporter
DigiCertEVcertificate
StatusPageStatuspage
top related