我的问题是。 问题,但我面临更多的制约因素:
是否有任何工具来做到这一点,或我更有理由打破雷克底和C#?
Im开放,可以指挥线或批量处理工具以及C/C#/D图书馆。
你们需要使用。
或许希望找到一个内容,使用Descendants ,然后获得InnerText。
该代码I今天用 up,将提取未经修改的三毫米案文。
public static string ExtractText(string html)
{
if (html == null)
{
throw new ArgumentNullException("html");
}
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(html);
var chunks = new List<string>();
foreach (var item in doc.DocumentNode.DescendantNodesAndSelf())
{
if (item.NodeType == HtmlNodeType.Text)
{
if (item.InnerText.Trim() != "")
{
chunks.Add(item.InnerText.Trim());
}
}
}
return String.Join(" ", chunks);
}
如果你想保持一定程度的格式,请在样本。
public string Convert(string path)
{
HtmlDocument doc = new HtmlDocument();
doc.Load(path);
StringWriter sw = new StringWriter();
ConvertTo(doc.DocumentNode, sw);
sw.Flush();
return sw.ToString();
}
public string ConvertHtml(string html)
{
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(html);
StringWriter sw = new StringWriter();
ConvertTo(doc.DocumentNode, sw);
sw.Flush();
return sw.ToString();
}
public void ConvertTo(HtmlNode node, TextWriter outText)
{
string html;
switch (node.NodeType)
{
case HtmlNodeType.Comment:
// don t output comments
break;
case HtmlNodeType.Document:
ConvertContentTo(node, outText);
break;
case HtmlNodeType.Text:
// script and style must not be output
string parentName = node.ParentNode.Name;
if ((parentName == "script") || (parentName == "style"))
break;
// get text
html = ((HtmlTextNode) node).Text;
// is it in fact a special closing node output as text?
if (HtmlNode.IsOverlappedClosingElement(html))
break;
// check the text is meaningful and not a bunch of whitespaces
if (html.Trim().Length > 0)
{
outText.Write(HtmlEntity.DeEntitize(html));
}
break;
case HtmlNodeType.Element:
switch (node.Name)
{
case "p":
// treat paragraphs as crlf
outText.Write("
");
break;
}
if (node.HasChildNodes)
{
ConvertContentTo(node, outText);
}
break;
}
}
private void ConvertContentTo(HtmlNode node, TextWriter outText)
{
foreach (HtmlNode subnode in node.ChildNodes)
{
ConvertTo(subnode, outText);
}
}
您可使用NUglify支持从超文本提取案文:
var result = Uglify.HtmlToText("<div> <p>This is <em> a text </em></p> </div>");
Console.WriteLine(result.Code); // prints: This is a text
由于它正在使用超文本5的习俗区,它应当相当有力(特别是如果文件没有包含任何错误),而且速度非常快(不涉及任何reg,而是纯粹的死胎区)。
我使用的是:
using System.Web;
public static string ExtractText(string html)
{
Regex reg = new Regex("<[^>]+>", RegexOptions.IgnoreCase);
string s =reg.Replace(html, " ");
s = HttpUtility.HtmlDecode(s);
return s;
}
如果你将超文本装入C#,然后使用C#/WinForms的Mshtml.dll或WebBrowser控制,那么你就可以把整个超文本文件当作树处理,把树林中捕获InnerText物体。
或者,你也可以使用文件。
例如:
WebBrowser webBrowser = new WebBrowser();
webBrowser.Url = new Uri("url_of_file"); //can be remote or local
webBrowser.DocumentCompleted += delegate
{
HtmlElementCollection collection = webBrowser.Document.All;
List<string> contents = new List<string>();
/*
* Adds all inner-text of a tag, including inner-text of sub-tags
* ie. <html><body><a>test</a><b>test 2</b></body></html> would do:
* "test test 2" when collection[i] == <html>
* "test test 2" when collection[i] == <body>
* "test" when collection[i] == <a>
* "test 2" when collection[i] == <b>
*/
for (int i = 0; i < collection.Count; i++)
{
if (!string.IsNullOrEmpty(collection[i].InnerText))
{
contents.Add(collection[i].InnerText);
}
}
/*
* <html><body><a>test</a><b>test 2</b></body></html>
* outputs: test test 2|test test 2|test|test 2
*/
string contentString = string.Join("|", contents.ToArray());
MessageBox.Show(contentString);
};
希望帮助!
以下是最佳方式:
public static string StripHTML(string HTMLText)
{
Regex reg = new Regex("<[^>]+>", RegexOptions.IgnoreCase);
return reg.Replace(HTMLText, "");
}
这里的一类是我为完成同样的事情而制定的。 所有现有的超声望图书馆都过于缓慢,而且速度太慢。 职能在法典评论中作了解释。 从我的基准来看,在亚马孙着陆页(包括以下)测试时,该代码比超声波包等值代码高出10X。
/// <summary>
/// The fast HTML text extractor class is designed to, as quickly and as ignorantly as possible,
/// extract text data from a given HTML character array. The class searches for and deletes
/// script and style tags in a first and second pass, with an optional third pass to do the same
/// to HTML comments, and then copies remaining non-whitespace character data to an ouput array.
/// All whitespace encountered is replaced with a single whitespace in to avoid multiple
/// whitespace in the output.
///
/// Note that the returned text content still may have named character and numbered character
/// references within that, when decoded, may produce multiple whitespace.
/// </summary>
public class FastHtmlTextExtractor
{
private readonly char[] SCRIPT_OPEN_TAG = new char[7] { < , s , c , r , i , p , t };
private readonly char[] SCRIPT_CLOSE_TAG = new char[9] { < , / , s , c , r , i , p , t , > };
private readonly char[] STYLE_OPEN_TAG = new char[6] { < , s , t , y , l , e };
private readonly char[] STYLE_CLOSE_TAG = new char[8] { < , / , s , t , y , l , e , > };
private readonly char[] COMMENT_OPEN_TAG = new char[3] { < , ! , - };
private readonly char[] COMMENT_CLOSE_TAG = new char[3] { - , - , > };
private int[] m_deletionDictionary;
public string Extract(char[] input, bool stripComments = false)
{
var len = input.Length;
int next = 0;
m_deletionDictionary = new int[len];
// Whipe out all text content between style and script tags.
FindAndWipe(SCRIPT_OPEN_TAG, SCRIPT_CLOSE_TAG, input);
FindAndWipe(STYLE_OPEN_TAG, STYLE_CLOSE_TAG, input);
if(stripComments)
{
// Whipe out everything between HTML comments.
FindAndWipe(COMMENT_OPEN_TAG, COMMENT_CLOSE_TAG, input);
}
// Whipe text between all other tags now.
while(next < len)
{
next = SkipUntil(next, < , input);
if(next < len)
{
var closeNext = SkipUntil(next, > , input);
if(closeNext < len)
{
m_deletionDictionary[next] = (closeNext + 1) - next;
WipeRange(next, closeNext + 1, input);
}
next = closeNext + 1;
}
}
// Collect all non-whitespace and non-null chars into a new
// char array. All whitespace characters are skipped and replaced
// with a single space char. Multiple whitespace is ignored.
var lastSpace = true;
var extractedPos = 0;
var extracted = new char[len];
for(next = 0; next < len; ++next)
{
if(m_deletionDictionary[next] > 0)
{
next += m_deletionDictionary[next];
continue;
}
if(char.IsWhiteSpace(input[next]) || input[next] == )
{
if(lastSpace)
{
continue;
}
extracted[extractedPos++] = ;
lastSpace = true;
}
else
{
lastSpace = false;
extracted[extractedPos++] = input[next];
}
}
return new string(extracted, 0, extractedPos);
}
/// <summary>
/// Does a search in the input array for the characters in the supplied open and closing tag
/// char arrays. Each match where both tag open and tag close are discovered causes the text
/// in between the matches to be overwritten by Array.Clear().
/// </summary>
/// <param name="openingTag">
/// The opening tag to search for.
/// </param>
/// <param name="closingTag">
/// The closing tag to search for.
/// </param>
/// <param name="input">
/// The input to search in.
/// </param>
private void FindAndWipe(char[] openingTag, char[] closingTag, char[] input)
{
int len = input.Length;
int pos = 0;
do
{
pos = FindNext(pos, openingTag, input);
if(pos < len)
{
var closenext = FindNext(pos, closingTag, input);
if(closenext < len)
{
m_deletionDictionary[pos - openingTag.Length] = closenext - (pos - openingTag.Length);
WipeRange(pos - openingTag.Length, closenext, input);
}
if(closenext > pos)
{
pos = closenext;
}
else
{
++pos;
}
}
}
while(pos < len);
}
/// <summary>
/// Skips as many characters as possible within the input array until the given char is
/// found. The position of the first instance of the char is returned, or if not found, a
/// position beyond the end of the input array is returned.
/// </summary>
/// <param name="pos">
/// The starting position to search from within the input array.
/// </param>
/// <param name="c">
/// The character to find.
/// </param>
/// <param name="input">
/// The input to search within.
/// </param>
/// <returns>
/// The position of the found character, or an index beyond the end of the input array.
/// </returns>
private int SkipUntil(int pos, char c, char[] input)
{
if(pos >= input.Length)
{
return pos;
}
do
{
if(input[pos] == c)
{
return pos;
}
++pos;
}
while(pos < input.Length);
return pos;
}
/// <summary>
/// Clears a given range in the input array.
/// </summary>
/// <param name="start">
/// The start position from which the array will begin to be cleared.
/// </param>
/// <param name="end">
/// The end position in the array, the position to clear up-until.
/// </param>
/// <param name="input">
/// The source array wherin the supplied range will be cleared.
/// </param>
/// <remarks>
/// Note that the second parameter is called end, not lenghth. This parameter is meant to be
/// a position in the array, not the amount of entries in the array to clear.
/// </remarks>
private void WipeRange(int start, int end, char[] input)
{
Array.Clear(input, start, end - start);
}
/// <summary>
/// Finds the next occurance of the supplied char array within the input array. This search
/// ignores whitespace.
/// </summary>
/// <param name="pos">
/// The position to start searching from.
/// </param>
/// <param name="what">
/// The sequence of characters to find.
/// </param>
/// <param name="input">
/// The input array to perform the search on.
/// </param>
/// <returns>
/// The position of the end of the first matching occurance. That is, the returned position
/// points to the very end of the search criteria within the input array, not the start. If
/// no match could be found, a position beyond the end of the input array will be returned.
/// </returns>
public int FindNext(int pos, char[] what, char[] input)
{
do
{
if(Next(ref pos, what, input))
{
return pos;
}
++pos;
}
while(pos < input.Length);
return pos;
}
/// <summary>
/// Probes the input array at the given position to determine if the next N characters
/// matches the supplied character sequence. This check ignores whitespace.
/// </summary>
/// <param name="pos">
/// The position at which to check within the input array for a match to the supplied
/// character sequence.
/// </param>
/// <param name="what">
/// The character sequence to attempt to match. Note that whitespace between characters
/// within the input array is accebtale.
/// </param>
/// <param name="input">
/// The input array to check within.
/// </param>
/// <returns>
/// True if the next N characters within the input array matches the supplied search
/// character sequence. Returns false otherwise.
/// </returns>
public bool Next(ref int pos, char[] what, char[] input)
{
int z = 0;
do
{
if(char.IsWhiteSpace(input[pos]) || input[pos] == )
{
++pos;
continue;
}
if(input[pos] == what[z])
{
++z;
++pos;
continue;
}
return false;
}
while(pos < input.Length && z < what.Length);
return z == what.Length;
}
}
HtmlAgility 包装:
// Where m_whitespaceRegex is a Regex with [s].
// Where sampleHtmlText is a raw HTML string.
var extractedSampleText = new StringBuilder();
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(sampleHtmlText);
if(doc != null && doc.DocumentNode != null)
{
foreach(var script in doc.DocumentNode.Descendants("script").ToArray())
{
script.Remove();
}
foreach(var style in doc.DocumentNode.Descendants("style").ToArray())
{
style.Remove();
}
var allTextNodes = doc.DocumentNode.SelectNodes("//text()");
if(allTextNodes != null && allTextNodes.Count > 0)
{
foreach(HtmlNode node in allTextNodes)
{
extractedSampleText.Append(node.InnerText);
}
}
var finalText = m_whitespaceRegex.Replace(extractedSampleText.ToString(), " ");
}
这里,您可以下载一个工具及其来源,即:。 页: 1
它包含一个超文本的帽子(这种东西显然必须比你的标准XML教区更宽容),而且你可以把超文本同XML相差得多。
如果你想要下载一个格式化产出的网页(即,没有超文本标签,而是出现在林克斯),那么:
lynx -dump URL > filename
如果该网页上有任何链接,则下载页末将列入这些链接的URLs。
http://www.perpetualpc.net/srtd_lynx.html“rel=“nofollow”>,可登录与-nolist的链接清单。 例如:
lynx -dump -nolist http://stackoverflow.com/a/10469619/724176 > filename
下一部法律
string? GetBodyPreview(string? htmlBody)
{
Regex reg = new Regex("<[^>]+>", RegexOptions.IgnoreCase);
htmlBody = reg.Replace(Crop(htmlBody, "<body ", 1000), "");
return Crop(HttpUtility.HtmlDecode(htmlBody), "", 255);
string Crop(string? text, string start, int maxLength)
{
var s = text?.IndexOf(start);
var r = (s >= 0 ? text?.Substring(text.IndexOf(start)) : text) ?? string.Empty;
return r.Substring(0, Int32.Min(r.Length, maxLength)).TrimStart();
}
}
What is the use of default keyword in C#? Is it introduced in C# 3.0 ?
I m the only developer in my company, and am getting along well as an autodidact, but I know I m missing out on the education one gets from working with and having code reviewed by more senior devs. ...
I m pretty new to the Objective-C world and I have a long history with .net/C# so naturally I m inclined to use my C# wits. Now here s the question: I feel really inclined to create some type of ...
I cannot figure out how to marshal a C++ CBitmap to a C# Bitmap or Image class. My import looks like this: [DllImport(@"test.dll", CharSet = CharSet.Unicode)] public static extern IntPtr ...
I have two EF entities. One has a property called HouseNumber. The other has two properties, one called StartHouseNumber and one called EndHouseNumber. I want to create a many to many association ...
How to user GhostScript DLL to convert PDF to PDF/A. I know I kind of have to call the exported function of gsdll32.dll whose name is gsapi_init_with_args, but how do i pass the right arguments? BTW, ...
Since I cannot order my dictionary, what is the best way of going about taking key value pairs and also maintaing an index?
Maybe it s something I m doing wrong. I m just learning Linq because I m bored. And so far so good. I made a little program and it basically just outputs all matches (foreach) into a label control. ...