Wiki source code of Filesystem Store
Last modified by Vincent Massol on 2025/11/17 17:19
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{box cssClass="floatinginfobox" title="**Contents**"}} | ||
| 2 | {{toc start="2"/}} | ||
| 3 | {{/box}} | ||
| 4 | |||
| 5 | By default, in XWiki the content of attachments and deleted attachments and documents are stored in what we call the "filesystem store". Its default location is ##<permanentdir>/store/file##. | ||
| 6 | |||
| 7 | {{version since="17.10.0RC1"}} | ||
| 8 | Despite its name, this "filesystem" store uses whatever [[blob store is configured>>documentation.xs.admin.store.blob.WebHome]] and can thus store attachments and deleted attachments and documents also in other blob stores like S3. The naming of the blobs is independent of the configured blob store. | ||
| 9 | {{/version}} | ||
| 10 | |||
| 11 | It's a simple file based storage in which documents and attachments references are (md5) hashed, to avoid problems with various limited (in terms of encoding and path size) file systems/blob stores. The filesystem store implements a two stage commit mechanism to maintain integrity even if the database fails to commit the attachment meta-data for example. | ||
| 12 | |||
| 13 | For example the attachment ##XWikiLogo.png## in document ##Sandbox.WebHome## is stored in the following location: ##/1/0/5d42329a923e687f5dff4887d80098/attachments/9/2/0bc685fa0da28168319c0126def81b##. | ||
| 14 | |||
| 15 | {{box title="And where is my entity located ?"}} | ||
| 16 | {{velocity}} | ||
| 17 | {{html}} | ||
| 18 | <form> | ||
| 19 | <input type="text" name="reference"#if($request.reference) value="$escapetool.xml($request.reference)#end"/> | ||
| 20 | <button name="serialize_document">Document</button> | ||
| 21 | <button name="serialize_attachment">Attachment</button> | ||
| 22 | </form> | ||
| 23 | {{/html}} | ||
| 24 | {{/velocity}} | ||
| 25 | |||
| 26 | {{groovy}} | ||
| 27 | def hash(str) | ||
| 28 | { | ||
| 29 | md5 = org.apache.commons.codec.digest.DigestUtils.md5Hex(str) | ||
| 30 | println "1. It's then md5 hashed (##${md5}##)." | ||
| 31 | |||
| 32 | char0 = md5.charAt(0) | ||
| 33 | println "1. A first folder is created with the first character of the hash (##${char0}##)." | ||
| 34 | |||
| 35 | char1 = md5.charAt(1) | ||
| 36 | println "1. A second folder is created with the second character of the hash (##${char1}##)." | ||
| 37 | |||
| 38 | charn = md5.substring(2) | ||
| 39 | println "1. A third folder is created with the remaining characters (##${charn}##)." | ||
| 40 | |||
| 41 | return "**$char0**/**$char1**/**$charn**" | ||
| 42 | } | ||
| 43 | |||
| 44 | def encodeDocument(documentReference) { | ||
| 45 | path = "" | ||
| 46 | if (documentReference.root.type == org.xwiki.model.EntityType.WIKI) { | ||
| 47 | println "1. A folder with the wiki name (##${documentReference.root.name}##) is created." | ||
| 48 | path = documentReference.root.name | ||
| 49 | } | ||
| 50 | |||
| 51 | localuid = services.component.getInstance(org.xwiki.model.reference.EntityReferenceSerializer.TYPE_STRING, "local/uid").serialize(documentReference) | ||
| 52 | println "1. The document reference is serialized using the local uid serializer (##${localuid}##)." | ||
| 53 | |||
| 54 | path += "/" + hash(localuid) | ||
| 55 | |||
| 56 | return path; | ||
| 57 | } | ||
| 58 | |||
| 59 | def encodeAttachment(attachmentReference) { | ||
| 60 | path = "" | ||
| 61 | if (reference.parent) { | ||
| 62 | path = encodeDocument(attachmentReference.parent) | ||
| 63 | } | ||
| 64 | |||
| 65 | path += "/attachments" | ||
| 66 | println "1. An ##attachments## folder is created." | ||
| 67 | |||
| 68 | name = attachmentReference.name | ||
| 69 | println "1. The attachment name (##$name##) is extracted from the reference." | ||
| 70 | |||
| 71 | path += "/" + hash(name) + "" | ||
| 72 | } | ||
| 73 | |||
| 74 | if (request.serialize_document != null) { | ||
| 75 | reference = services.component.getInstance(org.xwiki.model.reference.EntityReferenceResolver.TYPE_STRING, "relative").resolve(request.reference, org.xwiki.model.EntityType.DOCUMENT) | ||
| 76 | |||
| 77 | println "{{info}}" | ||
| 78 | path = encodeDocument(reference) | ||
| 79 | println "{{/info}}" | ||
| 80 | |||
| 81 | println "" | ||
| 82 | print "The final path of the folder which contains the content of this attachment is ##$path##." | ||
| 83 | } else if (request.serialize_attachment != null) { | ||
| 84 | reference = services.component.getInstance(org.xwiki.model.reference.EntityReferenceResolver.TYPE_STRING, "relative").resolve(request.reference, org.xwiki.model.EntityType.ATTACHMENT) | ||
| 85 | |||
| 86 | println "{{info}}" | ||
| 87 | path = encodeAttachment(reference) | ||
| 88 | println "{{/info}}" | ||
| 89 | |||
| 90 | println "" | ||
| 91 | print "The final path of the folder which contains the content of this document is ##$path##." | ||
| 92 | } | ||
| 93 | {{/groovy}} | ||
| 94 | {{/box}} | ||
| 95 | |||
| 96 | == Attachments and deleted attachments == | ||
| 97 | |||
| 98 | By default, inside the attachment folder, you will find both the current version of the attachment and its history. The file name is always ##f##, it's then optionally followed by the version (when it's a piece of the history) and the original file extension. | ||
| 99 | |||
| 100 | For example for an image of type png: | ||
| 101 | |||
| 102 | * ##f.png## | ||
| 103 | * ##fv1.1.png## | ||
| 104 | * ##fv2.1.png## | ||
| 105 | |||
| 106 | {{version since="16.4.0"}} | ||
| 107 | Most of the time, instead of the current version of the attachment, a "link" is created. It's a file which name is suffixed with ##.lnk## which contains the relative path it's representing. In standard condition, it always points to the latest version of the attachment. | ||
| 108 | |||
| 109 | For example for an image of type png: | ||
| 110 | |||
| 111 | * ##f.png.lnk## {{info}}contains the string ##fv2.1.png##{{/info}} | ||
| 112 | * ##fv1.1.png## | ||
| 113 | * ##fv2.1.png## | ||
| 114 | {{/version}} | ||
| 115 | |||
| 116 | The different between attachments and deleted attachment is the location of that folder inside the document folder: | ||
| 117 | |||
| 118 | * an attachment is located in ##/attachments/<hash/based/on the attachment name>/## | ||
| 119 | * a deleted is located in ##/deleted-attachments/<hash/based/on the attachment name>/<index of the deleted attachment>/## | ||
| 120 | |||
| 121 | == Deleted documents == | ||
| 122 | |||
| 123 | {{todo/}} |
