Module:Portal excerpt

ATTENTION! Template whitelisting or blacklisting is not just a "enter-and-be-done" deal. Whitelisting untested templates WILL cause problems!

Unknown templates that are not entered into its configuration are removed by default. This is expected behavior.

This module selects pages for portal excerpts. It can choose one page, a specified page, or random entry from a list, or choose a page linked from another page. Its excerpt processing is adapted from Module:Excerpt/portals and adapted for this very wiki. Selected page leads or sections are cleaned up and returned with an optional image and a link to the full article.

Usage

See the documentation on the five entry-point templates for more information.

The templates below currently do not call this module but its predecessor, as I haven't had time to test them fully yet.

{{#invoke:Portal excerpt|lead}}

Transcludes the lead of an article as an excerpt. See {{Transclude lead excerpt}} for more information.

{{#invoke:Portal excerpt|linked}}

Transcludes as an excerpt the lead of an article selected randomly from wikilinks on a page. See {{Transclude linked excerpt}} for more information.

{{#invoke:Portal excerpt|listitem}}

Transcludes as an excerpt the lead of an article selected randomly from list items on a page. See {{Transclude list item excerpt}} for more information.

{{#invoke:Portal excerpt|random}}

Transcludes as an excerpt the lead of an article randomly from the parameters. See {{Transclude random excerpt}} for more information.

{{#invoke:Portal excerpt|selected}}

Transcludes the lead of a selected article as an excerpt. See {{Transclude selected excerpt}} for more information.

Candidate selection

To learn more about a specific function (template), you are advised to read that template's /doc instead. The technical language here might be too confusing.

For lead, parameter 1 is the page to excerpt. For selected and random, the numbered parameters are candidate pages. selected requires selected= which may be a numbered or a named parameter. Numeric selections beyond the highest numbered parameter wrap around that number. random considers nonempty numbered parameters in numerical order.

For linked and listitem, parameter 1 is a source page containing links to candidate pages, not the page that will be excerpted. linked searches wikilinks in source wikitext. listitem searches wikilinks occurring on lines marked as bulletin or numbered list items, and will halt before a See also heading. Duplicate destinations will only be considered once. File, category, special, external, and other nonsensical destinations will be ignored.

The candidate fucntions try entries in random order until one produces a nonempty excerpt. Use nostubs=yes if you wish to skip pages containing {{Stub}} ({{Notastub}} is excused). A bad or empty candidate can be skipped while another candidate is tried. Because lead has only one candidate to try, if it fails you will meet an error excerpt. To see what this looks like, try pointing any of the templates at Schizophrenia with errors/debug enabled (more on this below).

Annotated links are also supported and recognized as links to target pages.

Candidate-specific options

An entry in a numbered candidate list may override some common settings:

{{#invoke:Portal excerpt|random

|Article A{{!}}paragraphs=1{{!}}files=1

|Article B{{!}}paragraphs=1-2{{!}}more=Read more

|Article C{{!}} ... {{!}} ...

}}

Supported per-candidate keys are paragraphs, files, fileargs, more, nobold, keepsubsections, captions, filelayout, and filecaption & filecaptionN along with multi- prefixed multiple-image keys (see more below). They override the corresponding general settings for that candidate. only cannot be set per candidate, it is global only, and a per-candidate only will be ignored. A candidate may begin with a wikilink, for example [[Article A|Display text]]|files=1 the link's target will be used as the page. Other keys embedded in candidate strings will be ignored.

Excerpt options

Parameter	Effect
paragraphs	Selects numbered paragraphs, for example 1, 1,3, or 1-2. By default all paragraphs in the chosen lead or section are included. A nonempty selection that matches no paragraphs makes that candidate fail.
files	Selects numbered images, for example 1 or 1,3-4. No images are included unless this is supplied. The module looks for existing directly linked images and images in supported infobox image fields, including galleries there. Repeated file names are included once.
filelayout=multiple	When two or more images are selected, setting filelayout to multiple wraps them in {{Multiple image}} instead of stacking separate [[File:...]] links. Has no effect with a single selected image. Layout parameters MUST be set with multi- prefixed keys, e.g. multi-width=, multi-align= etc. The exact list of supported keys can be found in the /config. If multi-align or a width option is not given, the layout borrows those from the first selected image's own fileargs. You should set those yourself if possible. Read {{Multiple image}} for more information regarding how to use this overall.
fileargs	Supplies pipe-separated formatting options for selected images, such as frameless{{!}}120px{{!}}right. Supported size, border, upright, format, and alignment options are applied when the file markup is rebuilt. The default format is thumb. Do NOT override thumb without good reason, as the excerpt will probably look ugly without it.
captions	Off by default. Set to yes to show an image's own original caption (its gallery line label, or the infobox's caption=). With captions off or unset, a selected image shows no caption unless filecaption/filecaptionN is given, those always apply regardless of this setting.
filecaption, filecaptionN	Replaces an image's original caption. filecaptionN applies to image number N and takes precedence over the general filecaption. An empty replacement removes the caption.
only=files or only=file	Returns selected images without excerpt text or the full-article link. NOTE: IMAGES COST EXPENSIVE PARSER CALLS. NOTE: GLOBAL ONLY.
nobold=yes	Removes bold markup from the excerpt text. Otherwise, a bold occurrence matching the selected page title is linked to that page.
more	Changes the final full-article link text. The default is (Full article...); more=no removes the link.
keepsubsections=yes	When excerpting a section, includes its nested subsections. By default, the excerpt stops at the next heading, even if that heading is a subsection.

Paragraph and file selections accept positive numbers through 20 and comma-separated ranges. An invalid item in a selection is ignored. The module does not display images by default, so specify files= when a portal card needs one. File options apply to the selected images, they never select images themselves.

Sections and links

Append #Section name to a candidate page to excerpt the section instead of the lead.

{{#invoke:Portal excerpt|lead|Example article#history|paragraphs=1-2}}

Keep in mind this has differing behavior by template.

For linked and listitem, use section= to restrict the source page from which links are collected. By default, nested subsections are searched too. sectiononly=yes stops at the next heading.

These source-selection options are separate from a candidate's own #Section name and keepsubsections setting.

Section calls, depending on how resolved, are handled in a hierarchy. Specifically, if you link a redirect page which redirects to a section, it will always override your specified section link.

Other candidates

Parameter	Effect
list=Title	Adds a collapsible list of the candidates that remain after the first successful choice. When no title is supplied, the title is Other articles.
showall=Separator	Appends excerpts for the remaining candidates after the first successful one. Its value is used literally as the separator between excerpts, so supply the wikitext you want between them. Enabling showall also enables the remaining-candidates list.

The first successful candidate is chosen randomly where there is more than one candidate. showall processes the remainder in their remaining order, and does not restore the original numbered order. The list likewise names the remaining candidate strings, not a fresh search of the source page.

With the current implementation, the retard who wrote this forgot to make list=no to suppress the list when showall is enabled, it will produce a box titled "no". This will be fixed soon. Use showall only where both the additional excerpts and the list are wanted.

Diagnostics

By default, error selections return no visible error message. Set errors=yes to display fatal errors, such as no valid candidates. Set debug=yes to display fatal errors and informational diagnostics, including skipped candidates and content removal logs. These settings help when figuring out how your candidate is processed and what was removed, etc.

Excerpt cleanup

This module removes the following from selected content:

1. References

2. Category links

3. Comments

4. TemplateStyle tags

5. choose blocks

6. original image markup

7. disallowed templates

8. parser functions

9. certain magic words

10. certain behavior switches

It only keeps authorized templates per set policy. Configuration is available in the /whitelist and /blacklist sections. This does not mean you can just add a template into a whitelist, doing so will cause issues. When files= is used, selected existing images are rebuilt separately from the excerpt text. If unsafe constructs somehow remain after cleanup, the candidate will be forced to fail and another can be tried.

See also

• Module:Excerpt/portals

Notes

---

--[[
-- Fork of WP module, adapted by User:Rickert / huge WIP
-- ATTENTION !
-- This version of Excerpt is designed specifically for the portal namespace and its associated templates
]]

-- @todo /config /whitelist /blacklist or something
-- @todo randomization issues

-- Templates listed here are prohibited in excerpts.
local config = mw.loadData('Module:Portal excerpt/config')
local whitelist = mw.loadData('Module:Portal excerpt/whitelist')
local blacklist = mw.loadData('Module:Portal excerpt/blacklist')
local errorMessages = config.errorMessages
local maxRedirects = config.maxRedirects
local maxSelectionIndex = config.maxSelectionIndex
local imageParams = config.imageParams
local multipleImageParams = config.multipleImageParams
local portableInfoboxTemplates = config.portableInfoboxTemplates
local fileTypes = config.fileTypes

local wantedTemplates = whitelist.wantedTemplates

local unwantedTemplates = blacklist.unwantedTemplates
local bannedTemplates = blacklist.bannedTemplates
local bannedMagic = blacklist.bannedMagic
local bannedSwitches = blacklist.bannedSwitches

-- templates listed here are wanted, but must be processed or meet criteria to stay in transclusion
local wantedCleanTemplates = {}
local templateCleaners = {}

local parser = require('Module:WikitextParser')
local escapeString = require('Module:String')._escapePattern
local yesno = require('Module:Yesno')
local p = {}

-- Helper function to test for truthy and falsy values
local function is(value)
	if value == nil or value == false or value == 0 then return false end

	if type(value) == 'string' then
		value = mw.ustring.lower(mw.text.trim(value))
		return value ~= '' and value ~= '0' and value ~= 'false' and value ~= 'no' and value ~= 'n' and value ~= 'off'
	end

	return true
end

local function normalizeTemplateName(name) -- normalize template name before applying policy
	if type(name) ~= 'string' then return '' end
	name = mw.ustring.gsub(name, '_', ' ')
	name = mw.text.trim(name)
	name = mw.ustring.gsub(name, '%s+', ' ')
	name = mw.ustring.gsub(name, '^[Tt][Ee][Mm][Pp][Ll][Aa][Tt][Ee]%s*:%s*', '')
	return name
end

-- Helper function to match from a list regular expressions
-- Like so: match pre..list[1]..post or pre..list[2]..post or ...
local function matchAny(text, pre, list, post, init)
	for _, item in ipairs(list) do
		local match = {mw.ustring.match(text, pre .. item .. post, init)}
		if match[1] then return unpack(match) end
	end
	return nil
end

local function cloneTable(value)
	if type(value) ~= 'table' then return value end

	local copy = {}
	for key, item in pairs(value) do
		copy[cloneTable(key)] = cloneTable(item)
	end

	return copy
end

-- Helper function to convert a comma-separated list of numbers or min-max ranges into a list of booleans
local function numberFlags(str)
	if str == nil then return {} end
	local flags = {}
	local ranges = mw.text.split(tostring(str), ',', true) -- parse ranges: "1,3-5" to {"1","3-5"}
	for _, r in ipairs(ranges) do
		local min, max = mw.ustring.match(r, '^%s*(%d+)%s*[-–—]%s*(%d+)%s*$') -- "3-5" to min=3 max=5
		if not max then
			local single = mw.ustring.match(r, '^%s*(%d+)%s*$')
			min, max = single, single
		end
		min = tonumber(min)
		max = tonumber(max)
		if min and max and min >= 1 and min <= max and max <= maxSelectionIndex then
			for index = min, max do
				flags[index] = true
			end
		end
	end
	return flags
end

-- Helper function to collect template arguments into an options table
local function parseArgs(frame)
	local args = {}
	local parent = frame:getParent()

	if parent then
		for key, value in pairs(parent.args or {}) do
			args[key] = value
		end
	end

	for key, value in pairs(frame.args or {}) do
		args[key] = value
	end -- args from a Lua call have priority over parent args from template
	args.paraflags = numberFlags(args['paragraphs'] or '')
	args.fileflags = numberFlags(args['files'] or '')
	return args
end

local function formatError(message, ...)
	message = errorMessages[message] or message or ''
	if select('#', ...) > 0 then message = mw.ustring.format(message, ...) end

	return message
end

local function luaError(code, ...)
	error({ excerptError = true, code = code, message = formatError(code, ...) }, 0)
end

local function isWikiError(err)
	return type(err) == 'table' and err.excerptError == true
end

local function formatDiagnostics(diagnostics, includeInfo)
	local lines = {}
	for _, entry in ipairs(diagnostics) do
		if entry.severity == 'fail' or includeInfo then table.insert(lines, '* ' .. entry.message) end
	end

	if #lines == 0 then return '' end

	return tostring(
		mw.html.create('div'):addClass('error'):wikitext(errorMessages.prefix .. '\n' .. table.concat(lines, '\n'))
	)
end

local function addFatal(diagnostics, code, ...)
	table.insert(diagnostics, {
		code = code,
		stage = 'fatal',
		severity = 'fail',
		message = formatError(code, ...),
	})
end

local function addMeds(diagnostics, severity, code, value, stage)
	if type(diagnostics) ~= 'table' then return end
	table.insert(diagnostics, {
		code = code,
		value = value,
		stage = stage,
		severity = severity,
		message = value ~= nil and formatError(code, value) or formatError(code),
	})
end

local function splitPageReference(page)
	if type(page) ~= 'string' then return nil, nil end

	page = mw.text.trim(page)
	if page == '' then return nil, nil end

	local pageName, section = mw.ustring.match(page, '^(.-)#(.*)$')
	if pageName == nil then pageName = page end

	pageName = mw.text.trim(pageName)
	if section ~= nil then section = mw.text.trim(section) end

	if pageName == '' then pageName = nil end
	if section == '' then section = nil end

	return pageName, section
end

-- @todo redirect -> page#snca loss fix
local function resolveTitle(page)
	if type(page) ~= 'string' then return luaError('noPage') end

	local requestedPage = mw.text.trim(page)
	if requestedPage == '' then return luaError('noPage') end

	local title = mw.title.new(requestedPage)
	if not title then return luaError('pageNotFound', requestedPage) end

	local visited = {}
	for _ = 0, maxRedirects do
		local key = title.prefixedText
		if visited[key] then return luaError('redirectLoop', requestedPage) end

		visited[key] = true

		local target = title.redirectTarget
		if not target then return title end

		title = target
	end

	return luaError('tooManyRedirects', requestedPage)
end

local function getContent(page)
	local title = assert(resolveTitle(page))
	local text = title:getContent()
	if text == nil then return luaError('pageNotFound', title.prefixedText) end

	local fragment = title.fragment
	if fragment == '' then fragment = nil end

	return text, title.prefixedText, fragment
end

local function normalizeSectionName(section)
	if type(section) ~= 'string' then return '' end
	-- uri decoding source headings here will ack valid plus signs
	section = mw.ustring.gsub(section, '_', ' ')
	section = mw.ustring.gsub(section, '%[%[[^%]|]-|(.-)%]%]', '%1')
	section = mw.ustring.gsub(section, '%[%[(.-)%]%]', '%1')
	section = mw.ustring.gsub(section, "''+", '')
	section = mw.ustring.gsub(section, '%s+', ' ')

	return mw.text.trim(section)
end

local function normalizeSectionReference(section)
	if type(section) ~= 'string' then return '' end

	section = mw.uri.decode(section, 'WIKI') -- preserve +
	return normalizeSectionName(section)
end

-- @todo if it turns out to be an issue, reintroduce header protection here, too bloaty otherwise

local function getHeadings(text)
	if type(text) ~= 'string' then return '', {} end
	local source = '\n' .. text .. '\n'
	local headings = {}
	local position = 1

	while true do
		local first, last, opening, name, closing =
			mw.ustring.find(source, '\n(==+)[ \t]*([^\n]-)[ \t]*(==+)[ \t]*\n', position)
		if not first then break end
		if #opening == #closing then
			table.insert(headings, {
				level = #opening,
				name = normalizeSectionName(name),
				headingStart = first,
				contentStart = last + 1,
			})
		end

		position = last
	end

	return source, headings
end

local function getLead(text)
	if type(text) ~= 'string' then return luaError('leadEmpty') end

	local source, headings = getHeadings(text)
	local contentEnd = mw.ustring.len(source)

	if headings[1] then contentEnd = headings[1].headingStart - 1 end

	local content = mw.text.trim(mw.ustring.sub(source, 1, contentEnd))
	if content == '' then return luaError('leadEmpty') end
	return content
end

local function getSection(text, section, keepSubsections)
	local sectionLabel = type(section) == 'string' and section or ''
	if type(text) ~= 'string' then return luaError('sectionNotFound', sectionLabel) end

	local requestedSection = normalizeSectionReference(sectionLabel)
	if requestedSection == '' then return luaError('sectionNotFound', sectionLabel) end

	keepSubsections = is(keepSubsections)
	local source, headings = getHeadings(text)
	local selectedIndex

	for index, heading in ipairs(headings) do
		if heading.name == requestedSection then
			selectedIndex = index
			break
		end
	end
	if not selectedIndex then return luaError('sectionNotFound', sectionLabel) end

	local selected = headings[selectedIndex]
	local contentEnd = mw.ustring.len(source)

	for index = selectedIndex + 1, #headings do
		local heading = headings[index]
		if not keepSubsections or heading.level <= selected.level then
			contentEnd = heading.headingStart - 1
			break
		end
	end
	local content = mw.text.trim(mw.ustring.sub(source, selected.contentStart, contentEnd))
	if content == '' then return luaError('sectionEmpty', sectionLabel) end

	return content
end

local function selectSource(page, options)
	options = options or {}

	local pageName, section = splitPageReference(page)
	if not pageName then return luaError('noPage') end

	local text, resolvedPage, redirectSection = getContent(pageName)
	if redirectSection then section = redirectSection end
	if section then
		text = getSection(text, section, options.keepsubsections)
	else
		text = getLead(text)
	end

	return text, resolvedPage, section
end

local function selectParagraphs(text, options)
	options = options or {}
	local paragraphs = parser.getParagraphs(text)
	local flags = options.paraflags or {}
	local selected = {}
	local hasSelection = false

	for _, enabled in pairs(flags) do
		if enabled then
			hasSelection = true
			break
		end
	end

	local requested = options.paragraphs ~= nil and mw.text.trim(tostring(options.paragraphs)) ~= ''

	if not hasSelection then
		if requested then return luaError('noSelectedParagraphs') end
		return table.concat(paragraphs, '\n\n')
	end

	for index, paragraph in ipairs(paragraphs) do
		if flags[index] then table.insert(selected, paragraph) end
	end

	if #selected == 0 then return luaError('noSelectedParagraphs') end

	return table.concat(selected, '\n\n')
end

local function findImageInTemplate(position, spans)
	for _, span in ipairs(spans) do
		if position >= span.first and position <= span.last then return true end
	end
	return false
end

local function collectDirectImages(text, unsafeSpans)
	if type(text) ~= 'string' then return {} end
	unsafeSpans = unsafeSpans or {}
	local images = {}
	local searchPosition = 1

	for _, fileLink in ipairs(parser.getFiles(text)) do
		local fileName = parser.getFileName(fileLink)
		local position = mw.ustring.find(text, escapeString(fileLink), searchPosition)
		if position then searchPosition = position + mw.ustring.len(fileLink) end

		if
			fileName
			and matchAny(fileName, '%.', fileTypes, '%s*$')
			and not (position and findImageInTemplate(position, unsafeSpans))
		then --skip file links in templates that will be stripped
			local title = mw.title.new(fileName, 6)
			if title and title.fileExists then
				table.insert(images, {
					name = title.text,
					link = fileLink,
					position = position or searchPosition,
					source = 'direct',
				})
			end
		end
	end

	return images
end

local galleryPipeMarker = '\27_GALLERY_PIPE_\27' -- dont touch unless you wanna break image processing

local function splitPipeSegments(text)
	local protected = text
	for _, nested in ipairs(parser.getLinks(text)) do
		local guarded = mw.ustring.gsub(nested, '|', galleryPipeMarker)
		protected = mw.ustring.gsub(protected, escapeString(nested), function()
			return guarded
		end, 1)
	end

	local segments = {}
	for segment in mw.text.gsplit(protected, '|', true) do
		segment = mw.ustring.gsub(segment, galleryPipeMarker, '|')
		table.insert(segments, segment)
	end
	return segments
end

local function classifyImageSegments(segments)
	local format, align, alt, link, caption
	for _, segment in ipairs(segments) do
		local trim = mw.text.trim(segment)
		local known = false

		if matchAny(trim, '^', imageParams[1], '$') then
			format = trim
			known = true
		elseif matchAny(trim, '^', imageParams[2], '$') then
			align = trim
			known = true
		elseif matchAny(trim, '^', imageParams[3], '$') then
			known = true
		elseif
			mw.ustring.match(trim, '^%d+%s*[Xx]%s*%d+%s*[Pp][Xx]$') or mw.ustring.match(trim, '^%d+%s*[Pp][Xx]$')
		then
			known = true
		else
			local key, value = mw.ustring.match(segment, '^%s*([%a][%w_%-]*)%s*=(.*)$')
			if key then
				local lowerKey = mw.ustring.lower(key)
				if lowerKey == 'alt' then
					alt = value
				elseif lowerKey == 'link' then
					link = value
				end
				known = true
			end
		end
		if not known then caption = segment end
	end
	return format, align, alt, link, caption
end

local function splitFileLink(link)
	local body = mw.ustring.match(link, '^%[%[(.-)%]%]$')
	if not body then return {} end

	local segments = splitPipeSegments(body)
	table.remove(segments, 1)
	return segments
end

local function parseTemplateParameters(template)
	local protectedTemplate = template
	for _, gallery in ipairs(parser.getGalleries(template)) do
		local protectedGallery = mw.ustring.gsub(gallery, '|', galleryPipeMarker)
		protectedTemplate = mw.ustring.gsub(protectedTemplate, escapeString(gallery), function()
			return protectedGallery
		end, 1)
	end
	local parameters, parameterOrder = parser.getTemplateParameters(protectedTemplate)
	for key, value in pairs(parameters) do
		if type(value) == 'string' then parameters[key] = mw.ustring.gsub(value, galleryPipeMarker, '|') end
	end
	return parameters, parameterOrder
end

local function parsePortableImageValue(value, fallbackCaption)
	if type(value) ~= 'string' then return {} end
	local images = {}
	local function addImage(fileName, caption, altParam, linkParam)
		fileName = mw.text.trim(fileName or '')
		fileName = mw.ustring.gsub(fileName, '^[Ff]ile%s*:%s*', '')
		fileName = mw.ustring.gsub(fileName, '^[Ii]mage%s*:%s*', '')

		if fileName == '' then return end
		if not matchAny(fileName, '%.', fileTypes, '%s*$') then
			local firstToken = mw.ustring.match(fileName, '^(%S+)')
			if firstToken and matchAny(firstToken, '%.', fileTypes, '%s*$') then
				fileName = firstToken -- rescue first file in case of unexpected landmine
			else
				return
			end
		end

		if type(caption) == 'string' then
			caption = mw.text.trim(caption)
			if caption == '' then caption = nil end
		else
			caption = nil
		end

		table.insert(images, {
			name = fileName,
			caption = caption,
			altParam = altParam,
			linkParam = linkParam,
		})
	end
	local galleries = parser.getGalleries(value)
	if #galleries > 0 then
		for _, gallery in ipairs(galleries) do
			local content = mw.ustring.gsub(gallery, '^<%s*[Gg][Aa][Ll][Ll][Ee][Rr][Yy][^>]*>', '', 1)
			content = mw.ustring.gsub(content, '</%s*[Gg][Aa][Ll][Ll][Ee][Rr][Yy]%s*>%s*$', '', 1)

			for line in mw.text.gsplit(content, '\n', true) do
				line = mw.text.trim(line)
				if line ~= '' then
					local segments = splitPipeSegments(line)
					local fileName = table.remove(segments, 1)
					local _, _, alt, link, caption = classifyImageSegments(segments)
					if caption == nil then caption = fallbackCaption end
					addImage(fileName, caption, alt, link)
				end
			end
		end
	elseif not mw.ustring.match(value, '^%s*%[%[') then
		addImage(value, fallbackCaption)
	end
	return images
end

-- @todo will this shit even work on every iteration of this spawn of satan? check
local function collectPortableInfoboxImages(text)
	if type(text) ~= 'string' then return {} end
	local images = {}
	local searchPosition = 1

	for _, template in ipairs(parser.getTemplates(text)) do
		local templateName = normalizeTemplateName(parser.getTemplateName(template))
		if matchAny(templateName, '^', portableInfoboxTemplates, '$') then
			local position = mw.ustring.find(text, escapeString(template), searchPosition)
			if position then searchPosition = position + mw.ustring.len(template) end
			local parameters = parseTemplateParameters(template)
			local normalizedParameters = {}
			for key, value in pairs(parameters) do
				if type(key) == 'string' then
					local normalizedKey = mw.ustring.lower(key)
					normalizedKey = mw.ustring.gsub(normalizedKey, '_', ' ')
					normalizedKey = mw.ustring.gsub(normalizedKey, '%s+', ' ')
					normalizedKey = mw.text.trim(normalizedKey)

					normalizedParameters[normalizedKey] = value
				end
			end
			local candidates = parsePortableImageValue(normalizedParameters.image, normalizedParameters.caption)
			for order, candidate in ipairs(candidates) do
				local title = mw.title.new(candidate.name, 6)
				if title and title.fileExists then
					table.insert(images, {
						name = title.text,
						caption = candidate.caption,
						altParam = candidate.altParam,
						linkParam = candidate.linkParam,
						position = position or mw.ustring.len(text) + 1,
						order = order,
						source = 'portable-infobox', -- hell
					})
				end
			end
		end
	end
	return images
end
local classifyTemplate

local function nukeImageinTemplate(text)
	local spans = {}
	local searchPosition = 1
	for _, template in ipairs(parser.getTemplates(text)) do
		local first, last = mw.ustring.find(text, escapeString(template), searchPosition)
		if first then searchPosition = last + 1 end
		local name = normalizeTemplateName(parser.getTemplateName(template))
		local policy = classifyTemplate(name)
		if
			first
			and (policy == 'banned' or policy == 'unwanted' or policy == 'unknown' or policy == 'portable-infobox')
		then
			table.insert(spans, { first = first, last = last })
		end
	end
	return spans
end

local function selectImages(text, options) -- expensive!!!
	options = options or {}
	local flags = options.fileflags or {}
	if next(flags) == nil then return {} end -- save calls
	local unsafeSpans = nukeImageinTemplate(text)
	local candidates = collectDirectImages(text, unsafeSpans)
	local portableImages = collectPortableInfoboxImages(text)

	for _, image in ipairs(portableImages) do
		table.insert(candidates, image)
	end
	table.sort(candidates, function(a, b)
		if a.position == b.position then return (a.order or 1) < (b.order or 1) end
		return a.position < b.position
	end)
	local unique = {}
	local seen = {}

	for _, candidate in ipairs(candidates) do
		if not seen[candidate.name] then
			seen[candidate.name] = true
			table.insert(unique, candidate)
		end
	end
	local selected = {}
	for index, candidate in ipairs(unique) do
		if flags[index] then
			local image = cloneTable(candidate)
			image.index = index
			table.insert(selected, image)
		end
	end

	return selected
end

function classifyTemplate(name)
	name = normalizeTemplateName(name)

	if name == '' then return 'unknown', name end
	if matchAny(name, '^', bannedTemplates, '$') then return 'banned', name end
	if matchAny(name, '^', unwantedTemplates, '$') then return 'unwanted', name end
	if matchAny(name, '^', portableInfoboxTemplates, '$') then return 'portable-infobox', name end
	if matchAny(name, '^', wantedCleanTemplates, '$') then return 'clean', name end
	if matchAny(name, '^', wantedTemplates, '$') then return 'wanted', name end

	return 'unknown', name
end

local cleanTemplates
local function cleanTemplate(template, diagnostics)
	if mw.ustring.sub(template, 1, 3) == '{{{' then
		addMeds(diagnostics, 'info', 'removedTemplateParameter', nil, 'templates')
		return ''
	end

	local name = parser.getTemplateName(template)
	name = normalizeTemplateName(name)
	if mw.ustring.sub(name, 1, 1) == '#' then
		local functionName = mw.ustring.match(name, '^#%s*([^:]+)') or name
		addMeds(diagnostics, 'info', 'removedParserFunction', functionName, 'templates')
		return ''
	end
	local upperName = mw.ustring.upper(name)
	if matchAny(upperName, '^', bannedMagic, '%s*:') or matchAny(upperName, '^', bannedMagic, '$') then
		addMeds(diagnostics, 'info', 'removedMagicWord', name, 'templates')
		return ''
	end
	local policy = classifyTemplate(name)
	if policy == 'banned' or policy == 'unwanted' or policy == 'portable-infobox' then
		addMeds(diagnostics, 'info', 'removedTemplate', name, 'templates')
		return ''
	end

	if policy == 'unknown' then
		addMeds(diagnostics, 'info', 'removedUnknownTemplate', name, 'templates')
		return ''
	end
	local body = mw.ustring.sub(template, 3, -3)
	local cleanedTemplate = '{{' .. cleanTemplates(body, diagnostics) .. '}}'
	if policy == 'wanted' then return cleanedTemplate end
	local cleaner = templateCleaners[name]
	if not cleaner then
		return luaError('missingTemplateCleaner', name) -- crash to avoid retards bypassing rules
	end
	local success, result = pcall(cleaner, cleanedTemplate, diagnostics)
	if not success or type(result) ~= 'string' then return luaError('templateCleanerFailed', name) end
	return result
end

cleanTemplates = function(text, diagnostics)
	if type(text) ~= 'string' then return '' end
	local cleaned = mw.ustring.gsub(text, '{(%b{})}', function(inner)
		return cleanTemplate('{' .. inner .. '}', diagnostics)
	end)
	return cleaned
end

local function removeBannedSwitches(text, diagnostics)
	if type(text) ~= 'string' then return '' end

	return mw.ustring.gsub(text, '__%s*([%a][%a%d_]*)%s*__', function(name)
		local normalizedName = mw.ustring.upper(name)

		for _, bannedName in ipairs(bannedSwitches) do
			if normalizedName == bannedName then
				addMeds(diagnostics, 'info', 'removedBehaviorSwitch', normalizedName, 'post-processing')
				return ''
			end
		end
		return nil
	end)
end

local function removeCats(text, diagnostics)
	if type(text) ~= 'string' then return '' end
	local categories = parser.getCategories(text)
	for _, category in ipairs(categories) do
		text = mw.ustring.gsub(text, escapeString(category), '', 1)
	end
	if #categories > 0 then addMeds(diagnostics, 'info', 'removedCats', nil, 'post-processing') end

	return text
end

local function removeComments(text, diagnostics)
	if type(text) ~= 'string' then return '' end
	local cleaned, count = mw.ustring.gsub(text, '<!%-%-.-%-%->', '')
	local cleanedFinal, unclosedCount = mw.ustring.gsub(cleaned, '<!%-%-.*$', '')
	if count > 0 or unclosedCount > 0 then addMeds(diagnostics, 'info', 'removedComments', nil, 'post-processing') end
	return cleanedFinal
end

local function removeRefs(text, diagnostics)
	if type(text) ~= 'string' then return '' end
	local references = parser.getReferences(text)
	local referenceLists = {}

	for _, tag in ipairs(parser.getTags(text)) do
		if parser.getTagName(tag) == 'references' then table.insert(referenceLists, tag) end
	end
	for _, referenceList in ipairs(referenceLists) do
		text = mw.ustring.gsub(text, escapeString(referenceList), '', 1)
	end
	for _, reference in ipairs(references) do
		text = mw.ustring.gsub(text, escapeString(reference), '', 1)
	end

	if #references > 0 or #referenceLists > 0 then
		addMeds(diagnostics, 'info', 'removedRefs', nil, 'post-processing')
	end

	return text
end

local function removeTemplateStyles(text, diagnostics)
	if type(text) ~= 'string' then return '' end
	local cleaned, count =
		mw.ustring.gsub(text, '<%s*[Tt][Ee][Mm][Pp][Ll][Aa][Tt][Ee][Ss][Tt][Yy][Ll][Ee][Ss]%s+[^>]-/%s*>', '')
	if count > 0 then addMeds(diagnostics, 'info', 'removedTemplateStyles', nil, 'post-processing') end
	return cleaned
end

local function removedChooseTags(text, diagnostics)
	if type(text) ~= 'string' then return '' end
	local cleaned, count =
		mw.ustring.gsub(text, '<%s*[Cc][Hh][Oo][Oo][Ss][Ee][^>]*>.-</%s*[Cc][Hh][Oo][Oo][Ss][Ee]%s*>', '')
	if count > 0 then addMeds(diagnostics, 'info', 'removedChoose', nil, 'post-processing') end
	return cleaned
end

local function removeImageMarkup(text)
	if type(text) ~= 'string' then return '' end
	for _, gallery in ipairs(parser.getGalleries(text)) do
		text = mw.ustring.gsub(text, escapeString(gallery), '', 1)
	end
	for _, file in ipairs(parser.getFiles(text)) do
		text = mw.ustring.gsub(text, escapeString(file), '', 1)
	end

	return text
end

local function validateFinalWikitext(text)
	if type(text) ~= 'string' then return luaError('failedContent', '') end

	if mw.ustring.find(text, '%{%{%{') then return luaError('unsafeTemplateParameterSurvived') end

	local parserFunction = mw.ustring.match(text, '{{%s*#%s*([^:|}]+)')
	if parserFunction then return luaError('unsafeParserFunctionSurvived', mw.text.trim(parserFunction)) end

	if mw.ustring.find(text, '<%s*[Tt][Ee][Mm][Pp][Ll][Aa][Tt][Ee][Ss][Tt][Yy][Ll][Ee][Ss]') then
		return luaError('unsafeTemplateStylesSurvived')
	end
	if mw.ustring.find(text, '<%s*[Cc][Hh][Oo][Oo][Ss][Ee]') then return luaError('unsafeChooseSurvived') end

	local function validateTemplates(source)
		for _, template in ipairs(parser.getTemplates(source)) do
			local name = normalizeTemplateName(parser.getTemplateName(template))
			local upperName = mw.ustring.upper(name)
			local policy = classifyTemplate(name)
			if policy ~= 'wanted' and policy ~= 'clean' then return luaError('unsafeTemplateSurvived', name) end
			if matchAny(upperName, '^', bannedMagic, '%s*:') or matchAny(upperName, '^', bannedMagic, '$') then
				return luaError('unsafeMagicWordSurvived', name)
			end
			validateTemplates(mw.ustring.sub(template, 3, -3))
		end
	end

	validateTemplates(text)
	for switch in mw.ustring.gmatch(text, '__%s*([%a][%w_]*)%s*__') do
		local upperSwitch = mw.ustring.upper(switch)
		for _, bannedSwitch in ipairs(bannedSwitches) do
			if upperSwitch == bannedSwitch then return luaError('unsafeBehaviorSwitchSurvived', upperSwitch) end
		end
	end

	return text
end

local function resolveImage(image, options, diagnostics)
	options = options or {}
	local alt, link, caption

	if image.link then
		local segments = splitFileLink(image.link)
		local _, _, segAlt, segLink, segCaption = classifyImageSegments(segments)
		alt, link, caption = segAlt, segLink, segCaption
	else
		alt = image.altParam
		link = image.linkParam
		caption = image.caption
	end
	if not is(options.captions) then caption = nil end

	local fileArgs = mw.text.trim(options.fileargs or '')
	local format, align
	local extraRaisin = {}
	if fileArgs ~= '' then
		local fileArgsSegments = {}
		for segment in mw.text.gsplit(fileArgs, '|', true) do
			table.insert(fileArgsSegments, segment)
		end
		format, align = classifyImageSegments(fileArgsSegments)
		for _, segment in ipairs(fileArgsSegments) do
			local trim = mw.text.trim(segment)
			if
				mw.ustring.match(trim, '^%d+%s*[Xx]%s*%d+%s*[Pp][Xx]$')
				or mw.ustring.match(trim, '^%d+%s*[Pp][Xx]$')
				or mw.ustring.match(trim, '^[Bb]order$')
				or mw.ustring.match(trim, '^[Uu]pright%s*=?%s*[%d%.]*$')
			then
				table.insert(extraRaisin, trim)
			end
		end
	end
	format = format or 'thumb'

	local overrideCaption = options.filecaption
	if image.index and options['filecaption' .. image.index] ~= nil then
		overrideCaption = options['filecaption' .. image.index]
	end
	if overrideCaption ~= nil then
		caption = mw.text.trim(tostring(overrideCaption))
		if caption == '' then caption = nil end
	end
	if caption then
		caption = cleanTemplates(caption, diagnostics)
		caption = mw.text.trim(caption)
		if caption == '' then caption = nil end
	end
	if caption then caption = validateFinalWikitext(caption) end

	if alt then
		alt = mw.text.trim(cleanTemplates(alt, diagnostics))
		if alt == '' then
			alt = nil
		else
			alt = validateFinalWikitext(alt)
		end
	end
	if link then
		link = mw.text.trim(cleanTemplates(link, diagnostics))
		if link == '' then
			link = nil
		else
			link = validateFinalWikitext(link)
		end
	end
	return {
		name = image.name,
		format = format,
		align = align,
		extras = extraRaisin,
		alt = alt,
		link = link,
		caption = caption,
	}
end

local function formatImage(resolved)
	local parts = { '[[File:' .. resolved.name, resolved.format }
	if resolved.align then table.insert(parts, resolved.align) end
	for _, extra in ipairs(resolved.extras) do
		table.insert(parts, extra)
	end
	if resolved.alt and mw.text.trim(resolved.alt) ~= '' then table.insert(parts, 'alt=' .. resolved.alt) end
	if resolved.link and mw.text.trim(resolved.link) ~= '' then table.insert(parts, 'link=' .. resolved.link) end
	if resolved.caption then table.insert(parts, resolved.caption) end

	return table.concat(parts, '|') .. ']]'
end

local function formatMultipleImage(resolved, options)
	local pixelParams = { width = true, total_width = true, image_gap = true }
	local params = {}
	for _, key in ipairs(multipleImageParams) do
		local value = options['multi-' .. key]
		if value ~= nil then
			value = mw.text.trim(tostring(value))
			if pixelParams[key] then value = mw.ustring.gsub(value, '%s*[Pp][Xx]$', '') end
			if value ~= '' then params[key] = value end
		end
	end

	local first = resolved[1]
	if not params.align and first.align then
		local align = mw.ustring.lower(mw.text.trim(first.align))
		if align == 'centre' then align = 'center' end -- fuck you
		params.align = align
	end
	if not params.width and not params.total_width then
		for _, extra in ipairs(first.extras) do
			local width = mw.ustring.match(extra, '^(%d+)%s*[Pp][Xx]$')
				or mw.ustring.match(extra, '^(%d+)%s*[Xx]%s*%d+%s*[Pp][Xx]$')
			if width then
				params.width = width
				break
			end
		end
	end
	local parts = { '{{Multiple image' }
	for _, key in ipairs(multipleImageParams) do
		if params[key] then table.insert(parts, key .. '=' .. params[key]) end
	end
	for index, entry in ipairs(resolved) do
		table.insert(parts, 'image' .. index .. '=' .. entry.name)
		if entry.alt and mw.text.trim(entry.alt) ~= '' then table.insert(parts, 'alt' .. index .. '=' .. entry.alt) end
		if entry.link and mw.text.trim(entry.link) ~= '' then
			table.insert(parts, 'link' .. index .. '=' .. entry.link)
		end
		if entry.caption then table.insert(parts, 'caption' .. index .. '=' .. entry.caption) end
	end

	return table.concat(parts, '|') .. '}}'
end

local function formatImages(images, options, diagnostics)
	options = options or {}
	local resolved = {}
	for _, image in ipairs(images) do
		table.insert(resolved, resolveImage(image, options, diagnostics))
	end

	local layout = mw.ustring.lower(mw.text.trim(tostring(options.filelayout or '')))
	if layout == 'multiple' and #resolved > 1 then return formatMultipleImage(resolved, options) end

	local output = {}
	for _, entry in ipairs(resolved) do
		table.insert(output, formatImage(entry))
	end
	return table.concat(output, '\n')
end

local function processExcerptContent(text, options, diagnostics)
	options = options or {}
	diagnostics = diagnostics or {}
	text = removeComments(text, diagnostics)
	text = removeTemplateStyles(text, diagnostics)
	text = removedChooseTags(text, diagnostics)
	local images = selectImages(text, options)
	text = removeRefs(text, diagnostics)
	text = removeCats(text, diagnostics)
	text = removeImageMarkup(text)
	text = cleanTemplates(text, diagnostics)
	text = selectParagraphs(text, options)
	text = removeBannedSwitches(text, diagnostics)
	text = mw.text.trim(text)
	text = validateFinalWikitext(text)
	return {
		text = text,
		images = images,
		formattedImages = formatImages(images, options, diagnostics),
		diagnostics = diagnostics,
	}
end

local function linkBoldTitle(text, pageTitle)
	if type(text) ~= 'string' or type(pageTitle) ~= 'string' or pageTitle == '' then return text end

	local function foldFirst(str)
		return mw.ustring.upper(mw.ustring.sub(str, 1, 1)) .. mw.ustring.sub(str, 2)
	end

	local foldedTitle = foldFirst(pageTitle)

	return (
		mw.ustring.gsub(text, "'''([^']+)'''", function(bold)
			if foldFirst(bold) == foldedTitle then return "'''[[" .. pageTitle .. '|' .. bold .. "]]'''" end
			return nil
		end)
	)
end

local function removeBold(text)
	if type(text) ~= 'string' then return text end
	return (mw.ustring.gsub(text, "'''(.-)'''", '%1'))
end

local function buildMoarLink(pageTitle, options)
	local more = options.more
	if more ~= nil and mw.ustring.lower(mw.text.trim(tostring(more))) == 'no' then return '' end
	if type(pageTitle) ~= 'string' or pageTitle == '' then return '' end

	local linkText = "'''(Full article...)'''"
	if type(more) == 'string' then
		local trim = mw.text.trim(more)
		if trim ~= '' then linkText = trim end
	end

	return '[[' .. pageTitle .. '|' .. linkText .. ']]'
end

local function assembleExcerpt(processed, options)
	options = options or {}

	if type(processed) ~= 'table' then return '' end

	local text = processed.text or ''
	local images = processed.formattedImages or ''
	if is(options.filesOnly) then return images end
	if is(options.nobold) then
		text = removeBold(text)
	else
		text = linkBoldTitle(text, processed.page)
	end
	local moreLink = buildMoarLink(processed.page, options)
	if moreLink ~= '' and text ~= '' then text = text .. '\n\n' .. moreLink end
	if images == '' then return text end
	if text == '' then return images end
	return images .. '\n' .. text
end

local function get(page, options, diagnostics)
	options = cloneTable(options or {})
	if type(options.paraflags) ~= 'table' then
		options.paraflags = numberFlags(options.paraflags or options.paragraphs)
	end

	if type(options.fileflags) ~= 'table' then options.fileflags = numberFlags(options.fileflags or options.files) end

	local only = mw.ustring.lower(mw.text.trim(tostring(options.only or '')))
	if only == 'file' or only == 'files' then options.filesOnly = true end
	local diagnostics = diagnostics or {}
	local text, resolvedPage, section = selectSource(page, options)
	local processed = processExcerptContent(text, options, diagnostics)

	processed.page = resolvedPage
	processed.section = section
	processed.output = assembleExcerpt(processed, options)
	return processed.output, processed
end

local function safeGet(page, options, diagnostics)
	diagnostics = diagnostics or {}
	local ok, output, processed = pcall(get, page, options, diagnostics)
	if ok then return true, output, processed, diagnostics end
	if isWikiError(output) then
		table.insert(diagnostics, {
			code = output.code,
			stage = 'fatal',
			severity = 'fail',
			message = output.message,
		})
		return false, '', nil, diagnostics
	end

	error(output, 0) -- not this thing's fault, let it go as a script error
end

function p.main(frame)
	local args = parseArgs(frame)
	local page = args[1] or args.page or args.article
	local diagnostics = {}
	local debug = is(args.debug)
	local showErrors = debug or is(args.errors)

	if type(page) ~= 'string' or mw.text.trim(page) == '' then
		addFatal(diagnostics, 'noPage')
		return showErrors and formatDiagnostics(diagnostics, debug) or ''
	end

	local ok, output = safeGet(page, args, diagnostics)
	local report = showErrors and formatDiagnostics(diagnostics, debug) or ''

	if ok then
		if output ~= '' and not mw.isSubsting() then output = frame:preprocess(output) end
		if report ~= '' then return output .. '\n' .. report end
		return output
	end

	return report
end

function p.get(page, options)
	return get(page, options)
end

local function parseCandidate(candidate, baseOptions)
	if type(candidate) ~= 'string' then return nil, nil end
	candidate = mw.text.trim(candidate)
	if candidate == '' then return nil, nil end
	local options = cloneTable(baseOptions or {})
	local page
	local argumentText
	local link

	link, argumentText = mw.ustring.match(candidate, '^(%[%b[]%])%s*(.*)$')
	if link then
		page = mw.ustring.match(link, '^%[%[%s*([^%]|]+)')
	else
		page, argumentText = mw.ustring.match(candidate, '^([^|]+)(.*)$')
	end
	if not page then return nil, nil end

	page = mw.text.trim(page)
	if page == '' then return nil, nil end
	argumentText = argumentText or ''
	argumentText = mw.ustring.gsub(argumentText, '^%s*|', '', 1)
	for _, argument in ipairs(mw.text.split(argumentText, '|', true)) do
		local key, value = mw.ustring.match(argument, '^%s*([^=]+)%s*=%s*(.-)%s*$')
		if key then
			key = mw.ustring.lower(mw.text.trim(key))
			if key == 'files' then
				options.files = value
				options.fileflags = numberFlags(value)
			elseif key == 'paragraphs' then
				options.paragraphs = value
				options.paraflags = numberFlags(value)
			elseif key == 'fileargs' then
				options.fileargs = value
			elseif key == 'more' then
				options.more = value
			elseif key == 'nobold' then
				options.nobold = value
			elseif key == 'keepsubsections' then
				options.keepsubsections = value
			elseif key == 'captions' then
				options.captions = value
			elseif key == 'filelayout' then
				options.filelayout = value
			elseif mw.ustring.match(key, '^filecaption%d*$') or mw.ustring.match(key, '^multi%-') then
				options[key] = value
			end
		end
	end
	return page, options
end

local function findStub(text)
	local hasStub, hasNotaStub = false, false
	for _, template in ipairs(parser.getTemplates(text)) do
		local name = normalizeTemplateName(parser.getTemplateName(template))
		if mw.ustring.match(name, '^[Ss]tub$') then
			hasStub = true
		elseif mw.ustring.match(name, '^[Nn]otastub$') then
			hasNotaStub = true
		end
	end
	return hasStub and not hasNotaStub
end

local function tryCandidates(candidates, baseOptions, diagnostics)
	baseOptions = baseOptions or {}
	diagnostics = diagnostics or {}
	local remaining = cloneTable(candidates)
	local output, processed

	while (not output or output == '') and #remaining > 0 do
		local index = #remaining > 1 and math.random(#remaining) or 1
		local candidateString = table.remove(remaining, index)
		local candidatePage, options = parseCandidate(candidateString, baseOptions)

		if candidatePage then
			local skip = false
			if is(baseOptions.nostubs) then
				local pageName = splitPageReference(candidatePage)
				local ok, rawText = pcall(getContent, pageName)
				if ok and findStub(rawText) then
					addMeds(diagnostics, 'info', 'candidateStub', candidatePage, 'candidates')
					skip = true
				end
			end
			if not skip then
				local ok, candidateOutput, candidateProcessed = pcall(get, candidatePage, options, diagnostics)
				if ok then
					if candidateOutput ~= '' then
						output = candidateOutput
						processed = candidateProcessed
					else
						addMeds(diagnostics, 'info', 'candidateEmpty', candidatePage, 'candidates')
					end
				elseif isWikiError(candidateOutput) then
					table.insert(diagnostics, {
						code = candidateOutput.code,
						stage = 'candidates',
						severity = 'info',
						message = candidateOutput.message,
					})
				else
					error(candidateOutput, 0)
				end
			end
		end
	end
	return output, processed, remaining
end

local function buildLeadCandidates(args)
	local page = args[1]
	if type(page) ~= 'string' or mw.text.trim(page) == '' then return {} end
	return { page }
end

local function buildSelectedCandidates(args)
	local count = 0
	for key in pairs(args) do
		if type(key) == 'number' and key > count then count = key end
	end
	local selectedKey = args.selected
	if selectedKey == nil then return {} end

	local page
	local numericKey = tonumber(selectedKey)
	if numericKey and count > 0 then
		numericKey = numericKey % count
		if numericKey == 0 then numericKey = count end
		page = args[numericKey]
	else
		page = args[selectedKey]
	end

	if type(page) ~= 'string' or mw.text.trim(page) == '' then return {} end
	return { page }
end

local function buildRandomCandidates(args)
	local keys = {}
	for key in pairs(args) do
		if type(key) == 'number' then table.insert(keys, key) end
	end
	table.sort(keys)

	local candidates = {}
	for _, key in ipairs(keys) do
		local value = args[key]
		if type(value) == 'string' and mw.text.trim(value) ~= '' then table.insert(candidates, value) end
	end
	return candidates
end

local function extractLinkCandidates(text, listItemsOnly)
	local candidates = {}
	local seen = {}
	local function addCandidate(candidate)
		candidate = mw.text.trim((mw.ustring.gsub(candidate, '^%s*:', '')))
		if candidate == '' or mw.ustring.sub(candidate, 1, 1) == '#' then return end
		local title = mw.title.new(candidate)
		if not title or title.isExternal or title.namespace < 0 or title.namespace == 6 or title.namespace == 14 then
			return
		end
		if seen[title.fullText] then return end
		seen[title.fullText] = true
		table.insert(candidates, candidate)
	end
	text = mw.ustring.gsub(text, '{{%s*[Aa]nnotated[ _]link%s*|%s*(.-)%s*}}', '[[%1]]') -- ?

	if listItemsOnly then
		text = mw.ustring.gsub(text, '\n==%s*[Ss]ee%s*[Aa]lso.*', '')
		local padded = '\n' .. text
		for candidate in mw.ustring.gmatch(padded, '\n:*[%*#][^\n]-%[%[%s*([^%]|\n]*)') do
			addCandidate(candidate)
		end
	else
		for candidate in mw.ustring.gmatch(text, '%[%[%s*([^%]|\n]*)') do
			addCandidate(candidate)
		end
	end

	return candidates
end

local function buildSourceCandidates(args, listItemsOnly, diagnostics)
	local source = args[1]
	if type(source) ~= 'string' or mw.text.trim(source) == '' then return {} end

	local ok, text = pcall(getContent, source)
	if not ok then
		if isWikiError(text) then
			if type(diagnostics) == 'table' then
				table.insert(diagnostics, {
					code = text.code,
					stage = 'candidates',
					severity = 'info',
					message = text.message,
				})
			end
			return {}
		end
		error(text, 0)
	end
	if is(args.section) then
		local ok2, sectionText = pcall(getSection, text, args.section, not is(args.sectiononly))
		if not ok2 then
			if isWikiError(sectionText) then
				if type(diagnostics) == 'table' then
					table.insert(diagnostics, {
						code = sectionText.code,
						stage = 'candidates',
						severity = 'info',
						message = sectionText.message,
					})
				end
				return {}
			end
			error(sectionText, 0)
		end
		text = sectionText
	end

	return extractLinkCandidates(text, listItemsOnly)
end

local function portal(frame, buildCandidates)
	local args = parseArgs(frame)
	local diagnostics = {}
	local debug = is(args.debug)
	local showErrors = debug or is(args.errors)

	local candidates = buildCandidates(args, diagnostics)
	local output, processed, remaining

	if #candidates > 0 then
		output, processed, remaining = tryCandidates(candidates, args, diagnostics)
	end
	remaining = remaining or {}

	if not output or output == '' then
		addFatal(diagnostics, 'noValidPages')
		return showErrors and formatDiagnostics(diagnostics, debug) or ''
	end

	if is(args.showall) then
		local separator = args.showall
		if type(separator) ~= 'string' or mw.text.trim(separator) == '' then separator = '{{clear}}{{hr}}' end
		for _, candidateString in ipairs(remaining) do
			local candidatePage, options = parseCandidate(candidateString, args)
			if candidatePage then
				local ok, extraOutput = pcall(get, candidatePage, options, diagnostics)
				if ok then
					if extraOutput ~= '' then output = output .. separator .. extraOutput end
				elseif isWikiError(extraOutput) then
					table.insert(diagnostics, {
						code = extraOutput.code,
						stage = 'candidates',
						severity = 'info',
						message = extraOutput.message,
					})
				else
					error(extraOutput, 0)
				end
			end
		end
	end
	if is(args.list) or (is(args.showall) and not is(args.list)) then
		local listTitle = args.list
		if type(listTitle) ~= 'string' or mw.text.trim(listTitle) == '' then listTitle = 'Other articles' end

		local items = {}
		for _, candidateString in ipairs(remaining) do
			local candidatePage = parseCandidate(candidateString, args)
			if candidatePage then table.insert(items, '[[:' .. candidatePage .. ']]') end
		end

		if #items > 0 then
			local box = '{{collapse top|title={{resize|85%|' .. listTitle .. '}}}}{{hlist'
			for _, item in ipairs(items) do
				box = box .. '|' .. item
			end
			box = box .. '}}\n{{collapse bottom}}'
			output = output .. box
		end
	end

	if not mw.isSubsting() then output = frame:preprocess(output) end
	local report = showErrors and formatDiagnostics(diagnostics, debug) or ''
	if report ~= '' then return output .. '\n' .. report end
	return output
end

function p.lead(frame)
	return portal(frame, buildLeadCandidates)
end
function p.selected(frame)
	return portal(frame, buildSelectedCandidates)
end
function p.random(frame)
	return portal(frame, buildRandomCandidates)
end
function p.linked(frame)
	return portal(frame, function(args, diagnostics)
		return buildSourceCandidates(args, false, diagnostics)
	end)
end
function p.listitem(frame)
	return portal(frame, function(args, diagnostics)
		return buildSourceCandidates(args, true, diagnostics)
	end)
end

p._test = { -- fill this in
	is = is,
	normalizeTemplateName = normalizeTemplateName,
	matchAny = matchAny,
	cloneTable = cloneTable,
	numberFlags = numberFlags,
	parseArgs = parseArgs,
	formatError = formatError,
	luaError = luaError,
	isWikiError = isWikiError,
	addMeds = addMeds,
	splitPageReference = splitPageReference,
	normalizeSectionName = normalizeSectionName,
	normalizeSectionReference = normalizeSectionReference,
	getLead = getLead,
	getSection = getSection,
	selectParagraphs = selectParagraphs,
}
return p