lunes, 4 de marzo de 2019

HtmlUnit(1): First HtmlUnit project

1. Create a Maven Project in Eclipse



In the top left menu select File-> New -> Maven -> Maven Project



Press Next

Press Next


Press Finish and a new project is created



2. The pom.xml file


The pom.xml is very simple, so it is added some stuff to look like this code. From the <properties> section on, is what should be added.


  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>ximodante</groupId>
  <artifactId>HTMLFillForms_MARIA_ANGELA</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>HTMLFillForms_MARIA_ANGELA</name>
  <description>HTMLFillForms</description>
  
  
  <properties>
    <cxf.version>3.2.6</cxf.version>
    <jax.version>2.3.0</jax.version>
    <lombok.version>1.18.4</lombok.version>
    <jackson.version>2.9.6</jackson.version>
    <h2.version>1.4.196</h2.version>
    <hibernate.version>5.2.16.Final</hibernate.version>
    
        
    <failOnMissingWebXml>false</failOnMissingWebXml>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <!-- Java version-->
    <java.version>10</java.version>
  </properties>  
  
  <dependencies>
  
    <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
      <version>${lombok.version}</version>
      <scope>provided</scope>
    </dependency>
    

    <!--  HTML Parser  -->
    <!--  See https://www.mkyong.com/java/how-to-automate-login-a-website-java-example/ -->
    <!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
    <dependency>
      <groupId>org.jsoup</groupId>
      <artifactId>jsoup</artifactId>
      <version>1.11.3</version>
    </dependency>
    

    <!-- Simulate a browser to fill forms !!! -->
    <!--  seehttps://stackoverflow.com/a/1383097/7704658   -->
    <!-- https://mvnrepository.com/artifact/net.sourceforge.htmlunit/htmlunit -->
    <dependency>
      <groupId>net.sourceforge.htmlunit</groupId>
      <artifactId>htmlunit</artifactId>
      <version>2.33</version>
    </dependency>
  
  
    <!--  JPA  -->
    <!-- MS-SQL Server JDBC JTDS bo de 2013-->
    <!-- https://mvnrepository.com/artifact/net.sourceforge.jtds/jtds -->
    <dependency>
      <groupId>net.sourceforge.jtds</groupId>
      <artifactId>jtds</artifactId>
      <version>1.3.1</version>
    </dependency>
     
       
    
    <!-- JPA 2.1 Provider -->
    <!-- https://mvnrepository.com/artifact/org.hibernate/hibernate-core -->
    <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-core</artifactId>
      <version>${hibernate.version}</version>
    </dependency>
    
    <!--  END JPA -->
  
    <!--BEGIN Java 9 references to JEE  not included in JDK9-->
    <!-- see http://openjdk.java.net/jeps/320-->
    
    <!-- https://mvnrepository.com/artifact/com.sun.xml.bind/jaxb-impl -->
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-impl</artifactId>
      <version>${jax.version}</version>
    </dependency>
        
    <!-- https://mvnrepository.com/artifact/com.sun.xml.bind/jaxb-core -->
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-core</artifactId>
      <version>${jax.version}</version>
    </dependency>
    
    
    <dependency>
      <groupId>com.sun.activation</groupId>
      <artifactId>javax.activation</artifactId>
      <version>1.2.0</version>
    </dependency>
    
    <dependency>
      <groupId>com.sun.xml.ws</groupId>
      <artifactId>jaxws-ri</artifactId>
      <version>${jax.version}</version>
      <type>pom</type>
    </dependency>
        
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-ri</artifactId>
      <version>${jax.version}</version>
      <type>pom</type>
    </dependency>
    
    <!-- End of Dependencies required for JAVA 9!!!! -->
    
  </dependencies>
  
  <!-- COMPILER PLUGIN -->
  <!-- This is OK  http://crunchify.com/how-to-create-build-java-project-including-all-dependencies-using-maven-maven-resources-maven-dependency-maven-jar-plugin-tutorial/
  -->
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.0</version>
          <configuration>
            <!-- release>10</release -->
            <source>10</source>
            <target>10</target>
            <showWarnings>true</showWarnings>
            <showDeprecation>true</showDeprecation>
          </configuration>
          
        </plugin>
      </plugins>
    </pluginManagement>
 
    <plugins>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <id>copy-resources</id>
            <phase>validate</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${basedir}/target/MyFolder</outputDirectory>
              <resources>
                <resource>
                  <directory>resources</directory>
                  <filtering>true</filtering>
                </resource>
              </resources>
            </configuration>
          </execution>
        </executions>
      </plugin>
 
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>3.1.1</version>
        <executions>
          <execution>
            <id>copy-dependencies</id>
            <phase>prepare-package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>${project.build.directory}/MyFolder/lib</outputDirectory>
              <overWriteReleases>false</overWriteReleases>
              <overWriteSnapshots>false</overWriteSnapshots>
              <overWriteIfNewer>true</overWriteIfNewer>
            </configuration>
          </execution>
        </executions>
      </plugin>
   
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.1.0</version>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <classpathPrefix>lib/</classpathPrefix>
              <mainClass>u.request.MyExecutableClass</mainClass>
            </manifest>
            <manifestEntries>
              <Class-Path>.</Class-Path>
            </manifestEntries>
          </archive>
 
          <finalName>MyFolder/MyJars</finalName>
        </configuration>
      </plugin>
    </plugins>
  
  </build>

The important thing is to add "htmlunit" dependency on line 49.

It is quite useful to use Lombok. So you have to follow the instructions to install it in your Eclipse IDE it and include the dependencies in the pom (line 28)

If you use Java 9 upwards you should include additional dependencies line 77

In this example, we are going to use JPA with some databases, as shown in line 56

3. A simple class


Let's create a package (for instance ximodante) inside "src/main/java" folder and create this class in it


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
package ximodante;

import java.io.IOException;
import java.net.MalformedURLException;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class HtmlUnit01 {

 public static void main(String[] args) {
  
  //01. Creatwe a WebClient object
  WebClient webClient=new WebClient();
  
  //02. Avoid certificate problems in https connection
  // @see https://stackoverflow.com/a/28543031/7704658
  boolean useInsecureSSL=true;
  webClient.getOptions().setUseInsecureSSL(useInsecureSSL);
  
  //03 Open a HTML page at an URL
  String anURL="https://www.google.com/";
  //String anURL="https://contrataciondelestado.es/wps/portal/organismosPublicos";
  HtmlPage page=null;
  try {
   page = webClient.getPage(anURL);
  } catch (FailingHttpStatusCodeException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  } catch (MalformedURLException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  } catch (IOException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
  
  //04. Print a text representation of the page
  System.out.println(page.asText());
 }
}

And after executing it, we get this output in the console


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
març 04, 2019 10:01:33 A. M. com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'https://www.google.com/' [1:11680] Error in expression. (Invalid token " ". Was expecting one of: <NUMBER>, "inherit", <IDENT>, <STRING>, <HASH>, <EMS>, <EXS>, <LENGTH_PX>, <LENGTH_CM>, <LENGTH_MM>, <LENGTH_IN>, <LENGTH_PT>, <LENGTH_PC>, <ANGLE_DEG>, <ANGLE_RAD>, <ANGLE_GRAD>, <TIME_MS>, <TIME_S>, <FREQ_HZ>, <FREQ_KHZ>, <RESOLUTION_DPI>, <RESOLUTION_DPCM>, <PERCENTAGE>, <DIMENSION>, <UNICODE_RANGE>, <URI>, <FUNCTION>, "progid:".)
març 04, 2019 10:01:33 A. M. com.gargoylesoftware.htmlunit.DefaultCssErrorHandler error
WARNING: CSS error: 'https://www.google.com/' [1:11704] Error in expression. (Invalid token " ". Was expecting one of: <NUMBER>, "inherit", <IDENT>, <STRING>, <HASH>, <EMS>, <EXS>, <LENGTH_PX>, <LENGTH_CM>, <LENGTH_MM>, <LENGTH_IN>, <LENGTH_PT>, <LENGTH_PC>, <ANGLE_DEG>, <ANGLE_RAD>, <ANGLE_GRAD>, <TIME_MS>, <TIME_S>, <FREQ_HZ>, <FREQ_KHZ>, <RESOLUTION_DPI>, <RESOLUTION_DPCM>, <PERCENTAGE>, <DIMENSION>, <UNICODE_RANGE>, <URI>, <FUNCTION>, "progid:".)
Google


GmailImágenes
Iniciar sesión



Buscar con GoogleVoy a tener suerte

Buscar con Google Voy a tener suerte

Ofrecido por Google en: català galego euskara English


España
PrivacidadCondicionesConfiguración
PublicidadEmpresaAcerca de


Happy coding!
Publicado por en

No hay comentarios:

Publicar un comentario

Suscribirse a: Enviar comentarios (Atom)