Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abandoned PR #12

Closed
wants to merge 41 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
b6fdff8
checkpoint commit of low GC Row...don't think its valid though
rymurr May 2, 2019
d38c007
working spark on 2.3 but not super fast
rymurr May 3, 2019
e67bb6d
moved to spark 2.4, started to add filter
rymurr May 5, 2019
78b114f
mostly add where clause
rymurr May 9, 2019
01701e0
start parallel read
rymurr May 22, 2019
de8e262
finished and working with dremio
rymurr Jun 21, 2019
bf69425
Create README.md
rymurr Oct 15, 2019
9b12364
Update README.md
rymurr Oct 15, 2019
408f0e2
license etc
Oct 15, 2019
f60e590
Update README.md
rymurr Oct 15, 2019
918405c
remove dremio specific stuff and bring up to date to 0.15.0
Oct 28, 2019
8b1f956
tested adn working on real spark
Dec 23, 2019
83892ea
fix tests
Jan 26, 2020
4392f61
relocate
Jan 26, 2020
91ffd8d
Cleanup and fixes for v1.0
Feb 17, 2020
bbaf8cf
Cleanup and fixes for v1.0
May 18, 2020
a7da568
Bump junit from 4.11 to 4.13.1
dependabot[bot] Oct 13, 2020
6d6d91d
Merge pull request #3 from rymurr/dependabot/maven/junit-junit-4.13.1
rymurr Oct 20, 2020
2452c50
add support for TimeStampMicroVector
cdoron Jan 14, 2021
c9cc3f2
Merge pull request #4 from cdoron/TimeStampMicroVector
Jan 20, 2021
0c23d97
Major refactor for spark 3; Compiling but test still failing.
kylebrooks-8451 Apr 13, 2022
8a2aaca
Getting futher in tests. Need to fix unserializable tasks.
kylebrooks-8451 Apr 13, 2022
13287d8
Further still need to debug serialization issues.
kylebrooks-8451 Apr 15, 2022
50768cb
Unit tests passing.
kylebrooks-8451 Apr 16, 2022
f492649
Fix lightScanBuilder resource leak.
kylebrooks-8451 Apr 16, 2022
882e090
Change to Java 11.
kylebrooks-8451 Apr 16, 2022
8400ca2
Upgrade maven shade plugin.
kylebrooks-8451 Apr 18, 2022
c7f1f85
Fix shaded jar includes.
kylebrooks-8451 Apr 19, 2022
0a3a3c9
Add full FlightClient options including client certs; Fix deprecated …
kylebrooks-8451 Apr 20, 2022
07c13c8
Implement Azure AD oauth client middleware
kylebrooks-8451 Apr 21, 2022
1fe1439
Bind local spark tests to localhost.
kylebrooks-8451 Apr 21, 2022
da8b92e
Add azure.identity to shaded jar.
kylebrooks-8451 Apr 21, 2022
552328a
Switch to simple token passing. Fix Postgres SQL syntax error with n…
kylebrooks-8451 Apr 26, 2022
c0edd48
Update CI to Java 11
kylebrooks-8451 Sep 23, 2022
b9b2150
Update license headers to submit back to open source upstream.
kylebrooks-8451 Sep 23, 2022
b42dcde
Update copyright and Authors.
kylebrooks-8451 Oct 11, 2022
c4c00d3
Exclude AUTHORS from license header check.
kylebrooks-8451 Oct 11, 2022
94725aa
Merge pull request #8 from kylebrooks-8451/master
kylebrooks-8451 Oct 11, 2022
98af23b
Initial maven-build workflow
mattphelps-8451 Oct 13, 2022
2ace064
Fix warning about ambiguous artifactId. See details:
mattphelps-8451 Oct 13, 2022
4fbb447
Skip Javadoc in maven build
mattphelps-8451 Oct 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
27 changes: 27 additions & 0 deletions .editorconfig
@@ -0,0 +1,27 @@
#
# Copyright (C) 2019 The flight-spark-source Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#


root = true

[*]
end_of_line = lf
insert_final_newline = true
indent_size = 2
indent_style = space

[*.js]
trim_trailing_whitespace = true
23 changes: 23 additions & 0 deletions .github/workflows/maven-build.yml
@@ -0,0 +1,23 @@
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven

name: Java CI with Maven

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: maven
- name: Build with Maven
run: mvn -B -V verify -Dmaven.javadoc.skip=true --file pom.xml
1 change: 1 addition & 0 deletions .gitignore
Expand Up @@ -2,6 +2,7 @@
.checkstyle
.classpath
.idea/
.vscode/
.project
.mvn/wrapper/maven-wrapper.jar
.profiler
Expand Down
30 changes: 30 additions & 0 deletions .mvn/extensions.xml
@@ -0,0 +1,30 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--

Copyright (C) 2019 The flight-spark-source Authors

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

-->
<extensions>
<extension>
<groupId>fr.jcgay.maven</groupId>
<artifactId>maven-profiler</artifactId>
<version>2.6</version>
</extension>
<extension>
<groupId>fr.jcgay.maven</groupId>
<artifactId>maven-notifier</artifactId>
<version>1.10.1</version>
</extension>
</extensions>
117 changes: 117 additions & 0 deletions .mvn/wrapper/MavenWrapperDownloader.java
@@ -0,0 +1,117 @@
/*
* Copyright (C) 2019 The flight-spark-source Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import java.net.*;
import java.io.*;
import java.nio.channels.*;
import java.util.Properties;

public class MavenWrapperDownloader {

private static final String WRAPPER_VERSION = "0.5.6";
/**
* Default URL to download the maven-wrapper.jar from, if no 'downloadUrl' is provided.
*/
private static final String DEFAULT_DOWNLOAD_URL = "https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/"
+ WRAPPER_VERSION + "/maven-wrapper-" + WRAPPER_VERSION + ".jar";

/**
* Path to the maven-wrapper.properties file, which might contain a downloadUrl property to
* use instead of the default one.
*/
private static final String MAVEN_WRAPPER_PROPERTIES_PATH =
".mvn/wrapper/maven-wrapper.properties";

/**
* Path where the maven-wrapper.jar will be saved to.
*/
private static final String MAVEN_WRAPPER_JAR_PATH =
".mvn/wrapper/maven-wrapper.jar";

/**
* Name of the property which should be used to override the default download url for the wrapper.
*/
private static final String PROPERTY_NAME_WRAPPER_URL = "wrapperUrl";

public static void main(String args[]) {
System.out.println("- Downloader started");
File baseDirectory = new File(args[0]);
System.out.println("- Using base directory: " + baseDirectory.getAbsolutePath());

// If the maven-wrapper.properties exists, read it and check if it contains a custom
// wrapperUrl parameter.
File mavenWrapperPropertyFile = new File(baseDirectory, MAVEN_WRAPPER_PROPERTIES_PATH);
String url = DEFAULT_DOWNLOAD_URL;
if(mavenWrapperPropertyFile.exists()) {
FileInputStream mavenWrapperPropertyFileInputStream = null;
try {
mavenWrapperPropertyFileInputStream = new FileInputStream(mavenWrapperPropertyFile);
Properties mavenWrapperProperties = new Properties();
mavenWrapperProperties.load(mavenWrapperPropertyFileInputStream);
url = mavenWrapperProperties.getProperty(PROPERTY_NAME_WRAPPER_URL, url);
} catch (IOException e) {
System.out.println("- ERROR loading '" + MAVEN_WRAPPER_PROPERTIES_PATH + "'");
} finally {
try {
if(mavenWrapperPropertyFileInputStream != null) {
mavenWrapperPropertyFileInputStream.close();
}
} catch (IOException e) {
// Ignore ...
}
}
}
System.out.println("- Downloading from: " + url);

File outputFile = new File(baseDirectory.getAbsolutePath(), MAVEN_WRAPPER_JAR_PATH);
if(!outputFile.getParentFile().exists()) {
if(!outputFile.getParentFile().mkdirs()) {
System.out.println(
"- ERROR creating output directory '" + outputFile.getParentFile().getAbsolutePath() + "'");
}
}
System.out.println("- Downloading to: " + outputFile.getAbsolutePath());
try {
downloadFileFromURL(url, outputFile);
System.out.println("Done");
System.exit(0);
} catch (Throwable e) {
System.out.println("- Error downloading");
e.printStackTrace();
System.exit(1);
}
}

private static void downloadFileFromURL(String urlString, File destination) throws Exception {
if (System.getenv("MVNW_USERNAME") != null && System.getenv("MVNW_PASSWORD") != null) {
String username = System.getenv("MVNW_USERNAME");
char[] password = System.getenv("MVNW_PASSWORD").toCharArray();
Authenticator.setDefault(new Authenticator() {
@Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password);
}
});
}
URL website = new URL(urlString);
ReadableByteChannel rbc;
rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(destination);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
fos.close();
rbc.close();
}

}
19 changes: 19 additions & 0 deletions .mvn/wrapper/maven-wrapper.properties
@@ -0,0 +1,19 @@
#
# Copyright (C) 2019 The flight-spark-source Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.zip
wrapperUrl=https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar

8 changes: 8 additions & 0 deletions .travis.yml
@@ -0,0 +1,8 @@
dist: focal
language: java
jdk: openjdk11
cache:
directories:
- $HOME/.m2
install: mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
script: mvn test -B
9 changes: 9 additions & 0 deletions AUTHORS
@@ -0,0 +1,9 @@
# This is the list of flight-spark-source's significant contributors.
#
# This does not necessarily list everyone who has contributed code,
# especially since many employees of one corporation may be contributing.
# To see the full list of contributors, see the revision history in
# source control.
Ryan Murray <rymurr@gmail.com>
Kyle Brooks <brookskd@gmail.com>
Doron Chen <cdoron@il.ibm.com>