pan-data/pd/DataMap

DataMap

class DataMap[K](df: DataFrame, groupCols: Array[String], map: Map[K, Array[Int]]) extends DataFrame

A DataMap is a DataFrame with a value-to-index mapping allowing fast hash-based access and grouped operations.

See also
Since

0.1.0

Note
  • Column operations return a DataMap[K] as a subtype of DataFrame.
  • You can drop the group columns but altering or replacing a group column results in an IllegalOperation exception (for methods which return the type DataMap[K]).
  • Note that index changing operations return a DataFrame since the mapping is no longer applicable to the resulting DataFrame.
Companion
object
class DataFrame
trait Dynamic
trait IndexOps[DataFrame]
class Object
trait Matchable
class Any

Value members

Concrete methods

@targetName("update")
override def &[T](namedSeries: (String, Series[T])): DataMap[K]
Definition Classes
@targetName("updateOperator")
override def &[T](series: Series[T]): DataMap[K]
Definition Classes
@targetName("prepend")
override def ::[T](series: Series[T]): DataMap[K]
Definition Classes
@targetName("prepend")
override def ::(df: DataFrame): DataMap[K]
Definition Classes
override def append[T](name: String, series: Series[T]): DataMap[K]
Definition Classes
override def append(series: Series[_]*): DataMap[K]
Definition Classes
@targetName("applyCols")
override def apply(cols: Seq[String]): DataMap[K]
Definition Classes
override def col[T](series: Series[T]): DataMap[K]
Definition Classes
override def col[T](col: String, series: Series[T]): DataMap[K]
Definition Classes
def dropGroupCols: DataMap[K]

Drops the group columns. The mapping of the values in the group columns to the rows is preserved.

Drops the group columns. The mapping of the values in the group columns to the rows is preserved.

Returns

DataMap without group columns.

Since

0.1.0

def groups: Groups[K]

Create groups from the DataMap which allow performing group-wise operations such as aggregation.

Create groups from the DataMap which allow performing group-wise operations such as aggregation.

Returns

Groups object.

See also
Since

0.1.0

def key(k: K): DataFrame

Selects a key in the DataMap.

Selects a key in the DataMap.

Value Params
k

Key value to match.

Returns

DataFrame for the selected key. If the key is not found, this DataFrame is empty.

Since

0.1.0

Note
  • Since only the index is set and the keys are hashed, this method is highly performant.
  • Note that the resulting DataFrame can have more than one row.
def keys: Iterable[K]

Keys in the map.

Keys in the map.

Returns

Iterable of keys.

Since

0.1.0

def sortByGroupSize: DataMap[K]

Sorts the DataFame with respect to the size of the groups staring with the largest group.

Sorts the DataFame with respect to the size of the groups staring with the largest group.

Returns

DataMap with sorted index.

Since

0.1.0

@targetName("sortValuesCols")
override def sortValues(cols: String*): DataMap[K]
Definition Classes
override def sortValues(keys: (String, Order)*): DataMap[K]
Definition Classes
override def sorted[T](col: String)(implicit ordering: Ordering[T]): DataMap[K]
Definition Classes
override def sorted[T1, T2](col1: String, col2: String)(implicit ordering1: Ordering[T1], ordering2: Ordering[T2]): DataFrame
Definition Classes
override def sorted[T1, T2, T3](col1: String, col2: String, col3: String)(implicit ordering1: Ordering[T1], ordering2: Ordering[T2], ordering3: Ordering[T3]): DataFrame
Definition Classes
override def sorted[T](key: (String, Order))(implicit ordering: Ordering[T]): DataMap[K]
Definition Classes
override def sorted[T1, T2](key1: (String, Order), key2: (String, Order))(implicit ordering1: Ordering[T1], ordering2: Ordering[T2]): DataFrame
Definition Classes
override def sorted[T1, T2, T3](key1: (String, Order), key2: (String, Order), key3: (String, Order))(implicit ordering1: Ordering[T1], ordering2: Ordering[T2], ordering3: Ordering[T3]): DataFrame
Definition Classes
override def update[T](col: String, series: Series[T]): DataMap[K]
Definition Classes
override def update[T](series: Series[T]): DataMap[K]
Definition Classes
override def update(series: Series[_]*): DataMap[K]
Definition Classes
@targetName("concat")
override def |[T](series: Series[T]): DataMap[K]
Definition Classes
@targetName("concat")
override def |[T](namedSeries: (String, Series[T])): DataMap[K]
Definition Classes
@targetName("concat")
override def |(df: DataFrame): DataMap[K]
Definition Classes

Inherited methods

def apply(series: Series[Boolean], col: String): Series[Any]

Extracts a column and slices the index by intersecting the current index with a boolean Series.

Extracts a column and slices the index by intersecting the current index with a boolean Series.

Value Params
col

Column name.

series

Boolean Series as mask, where only index positions kept that are true.

Returns

Series with sliced index.

Throws
ColumnNotFoundException

If the column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
def apply(array: Array[Int], col: String): Series[Any]

Extracts a column and slices the index by intersecting the current index with an array of index positions.

Extracts a column and slices the index by intersecting the current index with an array of index positions.

Value Params
array

Array of index positions.

col

Column name.

Returns

Series with sliced index.

Throws
ColumnNotFoundException

If the column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
def apply(seq: Seq[Int], col: String): Series[Any]

Extracts a column and slices the index by intersecting the current index with a sequence of index positions.

Extracts a column and slices the index by intersecting the current index with a sequence of index positions.

Value Params
col

Column name.

seq

Sequence of index positions.

Returns

Series with sliced index.

Throws
ColumnNotFoundException

If the column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
def apply(range: Range, col: String): Series[Any]

Extracts a column and slices the index by intersecting the current index with a range.

Extracts a column and slices the index by intersecting the current index with a range.

Value Params
col

Column name.

range

Range.

Returns

Series with sliced index.

Throws
ColumnNotFoundException

If the column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
def apply[T](row: Option[Int], col: String, default: => T)(implicit evidence$9: Typeable[T], evidence$10: ClassTag[T]): T

Returns a value for a columns and a row using a default value for undefined entries.

Returns a value for a columns and a row using a default value for undefined entries.

Value Params
col

Column name.

default

Default value for undefined values.

row

Row.

Returns

Value. Default value if row is None.

Throws
ColumnNotFoundException

If the column is not found.

IndexBoundsException

If row is not part of the base index.

See also
Since

0.1.0

Note

For an optimal performance in a loop, first extract the column as a Series.

Inherited from
DataFrame
def apply[T](row: Int, col: String, default: => T)(implicit evidence$7: Typeable[T], evidence$8: ClassTag[T]): T

Returns a value for a columns and a row using a default value for undefined entries.

Returns a value for a columns and a row using a default value for undefined entries.

Value Params
col

Column name.

default

Default value for undefined values.

row

Row.

Returns

Value.

Throws
ColumnNotFoundException

If the column is not found.

IndexBoundsException

If row is not part of the base index.

See also
Since

0.1.0

Note

For an optimal performance in a loop, first extract the column as a Series.

Inherited from
DataFrame
def apply[T](row: Option[Int], col: String)(implicit evidence$4: RequireType[T], evidence$5: Typeable[T], evidence$6: ClassTag[T]): Option[T]

Returns a value for a columns and a row.

Returns a value for a columns and a row.

Value Params
col

Column name.

row

Row.

Returns

Value as Option. None if row is None.

Throws
ColumnNotFoundException

If the column is not found.

IndexBoundsException

If row is not part of the base index.

See also
Since

0.1.0

Note

For an optimal performance in a loop, first extract the column as a Series.

Inherited from
DataFrame
def apply[T](row: Int, col: String)(implicit evidence$1: RequireType[T], evidence$2: Typeable[T], evidence$3: ClassTag[T]): Option[T]

Returns a value for a columns and a row.

Returns a value for a columns and a row.

Value Params
col

Column name.

row

Row.

Returns

Value as Option.

Throws
ColumnNotFoundException

If the column is not found.

IndexBoundsException

If row is not part of the base index.

See also
Since

0.1.0

Note

For an optimal performance in a loop, first extract the column as a Series.

Inherited from
DataFrame
def apply(col: String): Series[Any]

Returns a column as a Series.

Returns a column as a Series.

Value Params
col

Column name.

Returns

Column as a Series with the name of the column.

Throws
ColumnNotFoundException

If the column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
def apply(series: Series[Boolean]): DataFrame

Slices the index by intersecting the current index with a boolean Series.

Slices the index by intersecting the current index with a boolean Series.

Value Params
series

Boolean Series as mask, where only index positions kept that are true.

Returns

Object with sliced index and order of series.

See also
Since

0.1.0

Inherited from
IndexOps
def apply(array: Array[Int]): DataFrame

Slices the index by intersecting it with an array of index positions.

Slices the index by intersecting it with an array of index positions.

Value Params
array

Array of index positions.

Returns

Object with sliced index and order of array.

See also
Since

0.1.0

Inherited from
IndexOps
def apply(seq: Seq[Int]): DataFrame

Slices the index by intersecting it with a sequence of index positions.

Slices the index by intersecting it with a sequence of index positions.

Value Params
seq

Sequence of index positions.

Returns

Object with sliced index and order of seq.

See also
Since

0.1.0

Inherited from
IndexOps
def apply(range: Range): DataFrame

Slices the index by intersecting it with a range.

Slices the index by intersecting it with a range.

Value Params
range

Range.

Returns

Object with sliced index with ascending order.

See also
Since

0.1.0

Inherited from
IndexOps
def canEqual(a: Any): Boolean

Determines if an object is a DataFrame.

Determines if an object is a DataFrame.

Value Params
a

Any object.

Returns

True if the object is a DataFrame and false otherwise.

Since

0.1.0

Inherited from
DataFrame
def cols: DataFrameAppender

Appends (or replaces) multiple columns.

Appends (or replaces) multiple columns.

Returns

Appender object which appends Series on the right side.

Throws
MergeIndexException

If indices are not compatible.

See also
Since

0.1.0

Note
  • Existing columns with the same name are replaced by the rightmost column.
  • The index of the Series must be included in the left DataFrame.
  • Series might be copied if indices are not equivalent.
  • The index of the DataFrame is not altered.
  • For one column, this operation is equivalent to the | operator.
Example
 df.cols("price" -> Series(10.0, 20.0), "quantity" -> Series(5, 2))
 df.cols(Series(10.0, 20.0) as "price", Series(5, 2) as "quantity")
 df.cols(Series("price")(10.0, 20.0), Series("quantity")(5, 2))
 df.cols(price = Series(10.0, 20.0), quantity = Series(5, 2))
Inherited from
DataFrame
def columnArray: Array[Series[Any]]

All columns as array of Series.

All columns as array of Series.

Returns

Array with all columns as Series in defined order. All Series have the same index as the DataFrame.

Since

0.1.0

Inherited from
DataFrame
def columnIterator: Iterable[Series[Any]]

Iterates over all columns.

Iterates over all columns.

Returns

Iterable over all columns in defined order.

Since

0.1.0

Inherited from
DataFrame
def columns: Seq[String]

Sequence with column names.

Sequence with column names.

Returns

Sequence with column names in defined order.

Since

0.1.0

Inherited from
DataFrame
def contains(col: String): Boolean

Determines if a column is in the DataFrame.

Determines if a column is in the DataFrame.

Value Params
col

Column name.

Returns

True if DataFrame has the column col and false otherwise.

Since

0.1.0

Inherited from
DataFrame
def display(n: Int, width: Int, colWidth: Int): Unit

Prints the DataFrame as a table with an index column and annotated column types.

Prints the DataFrame as a table with an index column and annotated column types.

Value Params
colWidth

The width of each column.

n

The maximal numbers of rows.

width

The maximal width of a line.

See also
Since

0.1.0

Inherited from
DataFrame
def dropUndefined(cols: String*): DataFrame

Drops all rows with undefined (null) values with respect to specified columns.

Drops all rows with undefined (null) values with respect to specified columns.

Value Params
cols

Columns.

Returns

DataFrame restricted to rows without undefined values in columns cols.

Throws
ColumnNotFoundException

If one of the columns cols does not exist.

Since

0.1.0

Inherited from
DataFrame
def dropUndefined(): DataFrame

Drops all rows with undefined (null) values.

Drops all rows with undefined (null) values.

Returns

DataFrame restricted to rows without undefined values in all columns.

Since

0.1.0

Inherited from
DataFrame
override def equals(df: Any): Boolean

Determines if the object is a DataFrame and is equivalent. Equivalence implies:

Determines if the object is a DataFrame and is equivalent. Equivalence implies:

  • The same index. The indices are equal if they have the same elements, the same order and the same base index.
  • The column names are the same (but may have different order).
  • The values in all columns are equal.
Value Params
df

DataFrame (or other object) to compare to.

Returns

True if equal, false otherwise.

Since

0.1.0

Definition Classes
DataFrame -> Any
Inherited from
DataFrame
def get[T](row: Option[Int], col: String)(implicit evidence$14: RequireType[T], evidence$15: Typeable[T], evidence$16: ClassTag[T]): T

Returns a value for a columns and a row.

Returns a value for a columns and a row.

Value Params
col

Column name.

row

Row.

Returns

Value.

Throws
ColumnNotFoundException

If the column is not found.

NoSuchElementException

If the value is undefined, row is not in the index or row is None.

See also
Since

0.1.0

Note

For an optimal performance in a loop, first extract the column as a Series.

Inherited from
DataFrame
def get[T](row: Int, col: String)(implicit evidence$11: RequireType[T], evidence$12: Typeable[T], evidence$13: ClassTag[T]): T

Returns a value for a columns and a row.

Returns a value for a columns and a row.

Value Params
col

Column name.

row

Row.

Returns

Value.

Throws
ColumnNotFoundException

If the column is not found.

NoSuchElementException

If the value is undefined or row is not in the index.

See also
Since

0.1.0

Note

For an optimal performance in a loop, first extract the column as a Series.

Inherited from
DataFrame
def groupBy(cols: Seq[String]): Groups[Seq[Any]]

Groups the DataFrame by a sequence of columns (of arbitrary length).

Groups the DataFrame by a sequence of columns (of arbitrary length).

Value Params
cols

Sequence with column names.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupBy(col1: String, col2: String, col3: String): Groups[(Any, Any, Any)]

Groups the DataFrame by columns.

Groups the DataFrame by columns.

Value Params
col1

Column name.

col2

Column name.

col3

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupBy(col1: String, col2: String): Groups[(Any, Any)]

Groups the DataFrame by columns.

Groups the DataFrame by columns.

Value Params
col1

Column name.

col2

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupBy(col: String): Groups[Any]

Groups the DataFrame by a column.

Groups the DataFrame by a column.

Value Params
col

Column name.

Returns

Groups.

Throws
ColumnNotFoundException

If the a column does not exist.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByCol[T1, T2, T3](col1: String, col2: String, col3: String)(implicit evidence$23: Typeable[T1], evidence$24: ClassTag[T1], evidence$25: Typeable[T2], evidence$26: ClassTag[T2], evidence$27: Typeable[T3], evidence$28: ClassTag[T3]): Groups[(T1, T2, T3)]

Groups the DataFrame by typed columns.

Groups the DataFrame by typed columns.

Value Params
col1

Column name.

col2

Column name.

col3

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByCol[T1, T2](col1: String, col2: String)(implicit evidence$19: Typeable[T1], evidence$20: ClassTag[T1], evidence$21: Typeable[T2], evidence$22: ClassTag[T2]): Groups[(T1, T2)]

Groups the DataFrame by typed columns.

Groups the DataFrame by typed columns.

Value Params
col1

Column name.

col2

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByCol[T](col: String)(implicit evidence$17: Typeable[T], evidence$18: ClassTag[T]): Groups[T]

Groups the DataFrame by a typed column.

Groups the DataFrame by a typed column.

Value Params
col

Column name.

Returns

Groups.

Throws
ColumnNotFoundException

If the a column does not exist.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByColOption[T1, T2, T3](col1: String, col2: String, col3: String)(implicit evidence$35: Typeable[T1], evidence$36: ClassTag[T1], evidence$37: Typeable[T2], evidence$38: ClassTag[T2], evidence$39: Typeable[T3], evidence$40: ClassTag[T3]): Groups[(Option[T1], Option[T2], Option[T3])]

Groups the DataFrame by typed columns including undefined values.

Groups the DataFrame by typed columns including undefined values.

Value Params
col1

Column name.

col2

Column name.

col3

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByColOption[T1, T2](col1: String, col2: String)(implicit evidence$31: Typeable[T1], evidence$32: ClassTag[T1], evidence$33: Typeable[T2], evidence$34: ClassTag[T2]): Groups[(Option[T1], Option[T2])]

Groups the DataFrame by typed columns including undefined values.

Groups the DataFrame by typed columns including undefined values.

Value Params
col1

Column name.

col2

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByColOption[T](col: String)(implicit evidence$29: Typeable[T], evidence$30: ClassTag[T]): Groups[Option[T]]

Groups the DataFrame by a typed column including undefined values.

Groups the DataFrame by a typed column including undefined values.

Value Params
col

Column name.

Returns

Groups.

Throws
ColumnNotFoundException

If the a column does not exist.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByOption(cols: Seq[String]): Groups[Seq[Option[Any]]]

Groups the DataFrame by a sequence of columns (of arbitrary length) including undefined values.

Groups the DataFrame by a sequence of columns (of arbitrary length) including undefined values.

Value Params
cols

Sequence with column names.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByOption(col1: String, col2: String, col3: String): Groups[(Option[Any], Option[Any], Option[Any])]

Groups the DataFrame by columns including undefined values.

Groups the DataFrame by columns including undefined values.

Value Params
col1

Column name.

col2

Column name.

col3

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByOption(col1: String, col2: String): Groups[(Option[Any], Option[Any])]

Groups the DataFrame by columns including undefined values.

Groups the DataFrame by columns including undefined values.

Value Params
col1

Column name.

col2

Column name.

Returns

Groups.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def groupByOption(col: String): Groups[Option[Any]]

Groups the DataFrame by a column including undefined values.

Groups the DataFrame by a column including undefined values.

Value Params
col

Column name.

Returns

Groups.

Throws
ColumnNotFoundException

If the a column does not exist.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def head(n: Int): DataFrame

Head of object.

Head of object.

Value Params
n

Number of rows.

Returns

First n rows in index.

See also
Since

0.1.0

Inherited from
IndexOps
def indexBy(cols: Seq[String]): DataMap[Seq[Any]]

Indexes the DataFrame by a sequence of columns (of arbitrary length).

Indexes the DataFrame by a sequence of columns (of arbitrary length).

Value Params
cols

Sequence with column names.

Returns

DataMap.

Throws
IllegalOperation

If sequence of columns is empty.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexBy[T1, T2, T3](col1: String, col2: String, col3: String)(implicit evidence$47: Typeable[T1], evidence$48: ClassTag[T1], evidence$49: Typeable[T2], evidence$50: ClassTag[T2], evidence$51: Typeable[T3], evidence$52: ClassTag[T3]): DataMap[(T1, T2, T3)]

Indexes the DataFrame by (typed) columns.

Indexes the DataFrame by (typed) columns.

Value Params
col1

Column name.

col2

Column name.

col3

Column name.

Returns

DataMap.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexBy[T1, T2](col1: String, col2: String)(implicit evidence$43: Typeable[T1], evidence$44: ClassTag[T1], evidence$45: Typeable[T2], evidence$46: ClassTag[T2]): DataMap[(T1, T2)]

Indexes the DataFrame by (typed) columns.

Indexes the DataFrame by (typed) columns.

Value Params
col1

Column name.

col2

Column name.

Returns

DataMap.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexBy[T](col: String)(implicit evidence$41: Typeable[T], evidence$42: ClassTag[T]): DataMap[T]

Indexes the DataFrame by a (typed) column.

Indexes the DataFrame by a (typed) column.

Value Params
col

Column name.

Returns

DataMap.

Throws
ColumnNotFoundException

If the a column does not exist.

See also
Since

0.1.0

Note

Undefined (null) grouping values are ignored. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexByOption[T1, T2, T3](col1: String, col2: String, col3: String)(implicit evidence$59: Typeable[T1], evidence$60: ClassTag[T1], evidence$61: Typeable[T2], evidence$62: ClassTag[T2], evidence$63: Typeable[T3], evidence$64: ClassTag[T3]): DataMap[(Option[T1], Option[T2], Option[T3])]

Indexes the DataFrame by (typed) columns including undefined values.

Indexes the DataFrame by (typed) columns including undefined values.

Value Params
col1

Column name.

col2

Column name.

col3

Column name.

Returns

DataMap with Option keys.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexByOption[T1, T2](col1: String, col2: String)(implicit evidence$55: Typeable[T1], evidence$56: ClassTag[T1], evidence$57: Typeable[T2], evidence$58: ClassTag[T2]): DataMap[(Option[T1], Option[T2])]

Indexes the DataFrame by (typed) columns including undefined values.

Indexes the DataFrame by (typed) columns including undefined values.

Value Params
col1

Column name.

col2

Column name.

Returns

DataMap with Option keys.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexByOption[T](col: String)(implicit evidence$53: Typeable[T], evidence$54: ClassTag[T]): DataMap[Option[T]]

Groups the DataFrame by a (typed) column including undefined values.

Groups the DataFrame by a (typed) column including undefined values.

Value Params
col

Column name.

Returns

DataMap with Option keys.

Throws
ColumnNotFoundException

If the a column does not exist.

See also
Since

0.1.0

Note

Undefined (null) grouping are assembled in one group. Each Double.NaN value represents an individual group.

Inherited from
DataFrame
def indexIterator: Iterator[Int]

Iterator over the index.

Iterator over the index.

Returns

Iterator over the index in the current order.

Since

0.1.0

Inherited from
DataFrame
def info: String

Information string describing the DataFrame.

Information string describing the DataFrame.

Returns

Info string.

Since

0.1.0

Inherited from
DataFrame
def io: WriteAdapter

Adapter to write the DataFrame to a specific output. Use e.g.

Adapter to write the DataFrame to a specific output. Use e.g.

import pd.io.parquet.implicits

to import the respective format and read a DataFrame via

df.io.parquet.write(...)

import pd.io.parquet.implicits }}} respective format and read a DataFrame via

df.io.parquet.write(...)
Returns

WriteAdapter.

See also
Since

0.1.0

Inherited from
DataFrame
def joinInner(df: DataFrame, cols: String*): DataFrame

Performs an inner join equivalent to a SQL inner join with respect to the specified key columns.

Performs an inner join equivalent to a SQL inner join with respect to the specified key columns.

Value Params
cols

Key columns to be joined.

df

DataFrame to be joined.

Returns

The joined DataFrame.

Throws
IllegalOperation

If not at least one key column is specified.

See also
Since

0.1.0

Note
  • Columns in df are dropped that are not key columns but appear also in the original DataFrame. To keep these columns in the results, the columns must be renamed first.
  • The comparison of two undefined values (null) is false and does not satisfy a join condition.
Inherited from
DataFrame
def joinLeft(df: DataFrame, cols: String*): DataFrame

Performs a left join equivalent to a SQL outer left join with respect to the specified key columns.

Performs a left join equivalent to a SQL outer left join with respect to the specified key columns.

Value Params
cols

Key columns to be joined.

df

DataFrame to be joined.

Returns

The joined DataFrame.

Throws
IllegalOperation

If not at least one key column is specified.

See also
Since

0.1.0

Note
  • Columns in df are dropped that are not key columns but appear also in the original DataFrame. To keep these columns in the results, the columns must be renamed first.
  • The comparison of two undefined values (null) is false and does not satisfy a join condition.
Inherited from
DataFrame
def joinOuter(df: DataFrame, cols: String*): DataFrame

Performs an outer join equivalent to a SQL full outer join with respect to the specified key columns.

Performs an outer join equivalent to a SQL full outer join with respect to the specified key columns.

Value Params
cols

Key columns to be joined.

df

DataFrame to be joined.

Returns

The joined DataFrame.

Throws
IllegalOperation

If not at least one key column is specified.

See also
Since

0.1.0

Note
  • Columns in df are dropped that are not key columns but appear also in the original DataFrame. To keep these columns in the results, the columns must be renamed first.
  • The comparison of two undefined values (null) is false and does not satisfy a join condition.
Inherited from
DataFrame
def joinRight(df: DataFrame, cols: String*): DataFrame

Performs a right join equivalent to a SQL outer right join with respect to the specified key columns.

Performs a right join equivalent to a SQL outer right join with respect to the specified key columns.

Value Params
cols

Key columns to be joined.

df

DataFrame to be joined.

Returns

The joined DataFrame.

Throws
IllegalOperation

If not at least one key column is specified.

See also
Since

0.1.0

Note
  • Columns in df are dropped that are not key columns but appear also in the original DataFrame. To keep these columns in the results, the columns must be renamed first.
  • The comparison of two undefined values (null) is false and does not satisfy a join condition.
Inherited from
DataFrame
def merge(series: Series[_]*): DataFrame

Merges a Series into a DataFrame by index.

Merges a Series into a DataFrame by index.

Value Params
series

Series to concatenate.

Returns

DataFrame with all columns of df concatenated on the right side.

Since

0.1.0

Note
  • Columns with the same name are replaced by the rightmost column.
  • The order of index positions is only preserved if all indices are equal.
  • Use the join methods if you intend to join via column values.
Inherited from
DataFrame
def merge(df: DataFrame): DataFrame

Merges two DataFrame by index.

Merges two DataFrame by index.

Value Params
df

DataFrame to merge.

Returns

DataFrame with all columns of df concatenated on the right side.

Since

0.1.0

Note
  • Columns with the same name are replaced by the rightmost column.
  • The order of index positions is only preserved if the indices are equal.
  • Use the join methods if you intend to join via column values.
Inherited from
DataFrame
def numCols: Int

Number of columns.

Number of columns.

Returns

Number of columns.

Since

0.1.0

Inherited from
DataFrame
def numRows: Int

Number of rows.

Number of rows.

Returns

Length of the Series, i.e. number of elements in the index.

Since

0.1.0

Inherited from
DataFrame
def numRowsBase: Int

Number of rows of the underlying data vectors.

Number of rows of the underlying data vectors.

Returns

Number of elements in the base index.

Since

0.1.0

Inherited from
DataFrame
def plot: Plot

Creates a plot.

Creates a plot.

Returns

Plot object.

Since

0.1.0

Inherited from
DataFrame
def requires: Requirement

Requirement object which is used to throw exceptions if conditions are not met.

Requirement object which is used to throw exceptions if conditions are not met.

Returns

Requirement object.

Since

0.1.0

Inherited from
DataFrame
def resetIndex: DataFrame

Resets the index to a UniformIndex with index positions 0 to numRows - 1 while keeping the order of the elements. If the current index is not a uniform index, the columns are copied into new vectors with the order of the current index.

Resets the index to a UniformIndex with index positions 0 to numRows - 1 while keeping the order of the elements. If the current index is not a uniform index, the columns are copied into new vectors with the order of the current index.

Returns

DataFrame with a UniformIndex.

See also

sortIndex for sorting the index by index positions.

Since

0.1.0

Inherited from
DataFrame
@targetName("selectCols")
def select(cols: String*): DataFrame

Extracts DataFrame with selected columns.

Extracts DataFrame with selected columns.

Value Params
cols

Column names.

Returns

DataFrame with selected column.

Throws
ColumnNotFoundException

If a column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
@targetName("selectSeq")
def select(cols: Seq[String]): DataFrame

Extracts DataFrame with selected columns.

Extracts DataFrame with selected columns.

Value Params
cols

Column names.

Returns

DataFrame with selected column.

Throws
ColumnNotFoundException

If a column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
def select(cols: Array[String]): DataFrame

Extracts DataFrame with selected columns.

Extracts DataFrame with selected columns.

Value Params
cols

Column names.

Returns

DataFrame with selected column.

Throws
ColumnNotFoundException

If a column is not found.

See also
Since

0.1.0

Inherited from
DataFrame
@unused
def selectDynamic(col: String): Series[Any]

Selects a column via dot notation.

Selects a column via dot notation.

Value Params
col

Column name.

Returns

Column as a Series with the name of the column.

Throws
ColumnNotFoundException

If the column is not found.

See also
Since

0.1.0

Example
df.myColumn
Inherited from
DataFrame
def show(n: Int, width: Int, annotateIndex: Boolean, annotateType: Boolean, colWidth: Int): Unit

Prints the DataFrame as a table.

Prints the DataFrame as a table.

Value Params
annotateIndex

If true, the an index column is displayed.

annotateType

If true, the type for each column in displayed.

colWidth

The width of each column.

n

The maximal numbers of rows.

width

The maximal width of a line.

See also
Since

0.1.0

Inherited from
DataFrame
def sortIndex: DataFrame

Sorts the index (ascending).

Sorts the index (ascending).

Returns

Object with sorted index.

See also
Since

0.1.0

Inherited from
IndexOps
def tail(n: Int): DataFrame

Tail of object.

Tail of object.

Value Params
n

Number of rows.

Returns

Last n rows in index.

See also
Since

0.1.0

Inherited from
IndexOps
def toArray[T](col: String)(implicit evidence$65: RequireType[T], evidence$66: Typeable[T], evidence$67: ClassTag[T]): Array[Option[T]]

Copies a column into an array.

Copies a column into an array.

Value Params
col

Column name.

Returns

Array of type Option[T] with numRows elements.

See also
Since

0.1.0

Inherited from
DataFrame
def toFlatArray[T](col: String)(implicit evidence$68: RequireType[T], evidence$69: Typeable[T], evidence$70: ClassTag[T]): Array[T]

Copies a column into an array.

Copies a column into an array.

Value Params
col

Column name.

Returns

Array of type T.

See also
Since

0.1.0

Inherited from
DataFrame
def toFlatList[T](col: String)(implicit evidence$71: RequireType[T], evidence$72: Typeable[T], evidence$73: ClassTag[T]): List[T]

Copies a column into a List.

Copies a column into a List.

Value Params
col

Column name.

Returns

List of type T.

See also
Since

0.1.0

Inherited from
DataFrame
def toFlatSeq[T](col: String)(implicit evidence$74: RequireType[T], evidence$75: Typeable[T], evidence$76: ClassTag[T]): Seq[T]

Copies a column into a sequence.

Copies a column into a sequence.

Value Params
col

Column name.

Returns

Sequence of type T.

See also
Since

0.1.0

Inherited from
DataFrame
def toList[T](col: String)(implicit evidence$77: RequireType[T], evidence$78: Typeable[T], evidence$79: ClassTag[T]): List[Option[T]]

Copies a column into a List.

Copies a column into a List.

Value Params
col

Column name.

Returns

List of type List[T] with numRows elements.

See also
Since

0.1.0

Inherited from
DataFrame
def toSeq[T](col: String)(implicit evidence$80: RequireType[T], evidence$81: Typeable[T], evidence$82: ClassTag[T]): Seq[Option[T]]

Copies a column into a sequence.

Copies a column into a sequence.

Value Params
col

Column name.

Returns

Sequence of type Seq[T] with numRows elements.

See also
Since

0.1.0

Inherited from
DataFrame
def toString(n: Int, width: Int, annotateIndex: Boolean, annotateType: Boolean, colWidth: Int, indexWidth: Int): String

Renders the DataFrame as a table.

Renders the DataFrame as a table.

Value Params
annotateIndex

If true, the an index column is displayed.

annotateType

If true, the type for each column in displayed.

colWidth

The width of each column.

indexWidth

The width of the index colum.

n

The maximal numbers of rows.

width

The maximal width of a line.

Returns

Formatted table.

Since

0.1.0

Inherited from
DataFrame
override def toString: String

Renders the DataFrame as a table using default parameters.

Renders the DataFrame as a table using default parameters.

Returns

Formatted table.

Since

0.1.0

Definition Classes
DataFrame -> Any
Inherited from
DataFrame
def union(df: DataFrame*): DataFrame

Appends row-wise one or multiple DataFrame objects. The method materializes all indices into a uniform index.

Appends row-wise one or multiple DataFrame objects. The method materializes all indices into a uniform index.

Value Params
df

DataFrame object to be appended.

Returns

DataFrame with exactly the same columns.

Throws
ColumnNotFoundException

If the DataFrame objects to be appended don't have the same columns.

SeriesCastException

If the underlying types of the columns do not match.

Since

0.1.0

Inherited from
DataFrame
def valueCounts(cols: Seq[String], countCol: String, dropUndefined: Boolean, order: Order, asFraction: Boolean): DataFrame

Counts the number of rows for unique value pairs in key columns.

Counts the number of rows for unique value pairs in key columns.

Value Params
asFraction

If true, it returns the total fraction as Double for an unique key value relative to the total number of rows (including undefined values). Otherwise the number of rows is returned as Int column.

cols

Key columns.

countCol

Name of the resulting column.

dropUndefined

If true, drops rows with undefined values in a key column (the default includes undefined values).

order

Order of the countCol column.

Returns

DataFrame with the key columns and the countCol column.

Since

0.1.0

Inherited from
DataFrame
def valueCounts(cols: String*): DataFrame

Counts the number of rows for unique value pairs in key columns (including undefined key values).

Counts the number of rows for unique value pairs in key columns (including undefined key values).

Value Params
cols

Key columns.

Returns

DataFrame with the key columns and a "count" column.

Since

0.1.0

Inherited from
DataFrame